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Why GAO Did This Study 



What GAO Found 



Explosives represent a continuing 
threat to aviation security. The 
Transportation Security Administration 
(TSA), within the Department of 
Homeland Security (DHS), seeks to 
ensure through the Electronic Baggage 
Screening Program (EBSP) that 
checked-baggage-screening 
technology is capable of detecting 
explosives. Generally, the explosives 
detection system (EDS) is used in 
conjunction with explosives trace 
detection (ETD) machines to identify 
and resolve threats in checked 
baggage. As requested, GAO 
assessed the extent to which: (1 ) TSA 
revised explosives detection 
requirements and deployed technology 
to meet those requirements, and (2) 
TSA's approach to the current EDS 
acquisition meets best practices for 
schedules and cost estimates and 
includes plans for potential upgrades of 
deployed EDSs. GAO analyzed EDS 
requirements, compared the EDS 
acquisition schedule against GAO best 
practices, and interviewed DHS 
officials. This is a public version of a 
sensitive report that GAO issued in 
May 2011. 

What GAO Recommends 

GAO recommends that TSA, among 
other things, develop a plan to ensure 
that new machines, as well as those 
machines currently deployed in 
airports, will be operated at the levels 
in established requirements, collect 
explosives data before initiating new 
procurements, and develop a reliable 
schedule for the EBSP. DHS 
concurred with all of GAO's 
recommendations and has initiated 
actions to implement them. 



TSA revised EDS explosives detection requirements in January 2010 to better 
address current threats and plans to implement these requirements in a phased 
approach. The first phase, which includes implementation of the previous 2005 
requirements, is to take years to fully implement. However, deploying EDSs that 
meet 2010 requirements could prove difficult given that TSA did not begin 
deployment of EDSs meeting 2005 requirements until 4 years later in 2009. As of 
January 201 1 , some number of the EDSs in TSA's fleet are detecting explosives 
at the level established in 2005. The remaining EDSs in the fleet are configured 
to meet the 1998 requirements because TSA either has not activated the 
included software or has not installed the needed hardware and software to allow 
these EDSs to meet the 2005 requirements. Developing a plan to deploy and 
operate EDSs to meet the most recent requirements could help ensure EDSs are 
operating most effectively and should improve checked-baggage screening. 
However, TSA has faced challenges in procuring the first 260 EDSs to meet 
2010 requirements. For example, due to the danger associated with some 
explosives, TSA and DHS encountered challenges in developing simulants and 
collecting data on the explosives' physical and chemical properties needed by 
vendors and agencies to develop detection software and test EDSs prior to the 
current acquisition. Also, TSA's decision to pursue EDS procurement during data 
collection complicated both efforts and resulted in a delay of over 7 months for 
the current acquisition. Completing data collection for each phase of the 2010 
requirements prior to pursuing EDS procurements that meet those requirements 
could help TSA avoid additional schedule delays. 

TSA has established a schedule for the current EDS acquisition, but it does not 
fully comply with best practices, and TSA has not developed a plan to upgrade its 
EDS fleet. For example, the schedule is not reliable because it does not reflect all 
planned program activities and does not include a timeline to deploy EDSs or 
plans to procure EDSs to meet subsequent phases of the 2010 requirements. 
Developing a reliable schedule would help TSA better monitor and oversee the 
progress of the EDS acquisition. TSA officials stated that to meet the 2010 
requirements, TSA will likely upgrade many of the current fleet of EDSs as well 
as the first 260 EDS machines to be purchased under the current acquisition. 
However, TSA has no plan in place outlining how it will approach these 
upgrades. Because TSA is implementing the 2010 requirements in a phased 
approach, the same EDS machines may need to be upgraded multiple times. 
TSA officials stated that they were confident the upgrades could be completed on 
deployed machines. However, without a plan, it will be difficult for TSA to provide 
reasonable assurance that the upgrades will be feasible or cost-effective. 
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The Honorable John L. Mica 
Chairman 

Committee on Transportation 

and Infrastructure 
House of Representatives 

The Honorable Charles W. Dent 
House of Representatives 

Explosives represent a continuing threat to aviation security, according to 
the Department of Homeland Security (DHS). The Transportation Security 
Administration (TSA), the agency within DHS responsible for overseeing 
and ensuring civil aviation security, is seeking to ensure that the 
technology it uses to screen checked baggage is capable of detecting 
these threats. 

TSA's Electronic Baggage Screening Program (EBSP), one of the largest 
acquisition programs within DHS, certifies and acquires systems used to 
screen checked baggage at 462 commercial airports throughout the 
United States. 1 TSA certifies explosives detection-screening technologies 
to ensure they meet explosives detection requirements developed in 
conjunction with the DHS Science and Technology Directorate (S&T) 
along with input from other agencies, such as the Federal Bureau of 
Investigation and Department of Defense (DOD). S&T conducts research 
and development of new technologies while its Transportation Security 
Laboratory (TSL) is responsible for, among other things, independent test 
and evaluation of new technologies, including conducting certification 
testing of checked-baggage-screening technologies. 

Pursuant to the Aviation and Transportation Security Act (ATSA), enacted 
in November 2001, TSA deploys explosives detection systems (EDS) and 



1 As of January 201 1 , TSA provides for, or oversees the provision of, screening and other 
security activities at 462 airports in the United States that operate under TSA-approved 
security programs pursuant to 49 C.F.R. part 1542, referred to in this report as 
"commercial airports." At some of these airports, a private-screening company under 
contract to TSA and subject to TSA oversight carries out the screening function using 
private screeners through TSA's Screening Partnership Program. See 49 U.S.C. § 44920. 
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explosives trace detection (ETD) machines to screen all checked 
baggage transported by U.S. and foreign air carriers departing from U.S. 
commercial airports. An EDS machine uses computed tomography (CT) 
technology to automatically measure the physical characteristics of 
objects in baggage. The system automatically triggers an alarm when 
objects that exhibit the physical characteristics of explosives are detected. 
An ETD machine is used to chemically analyze trace materials after a 
human operator swabs checked baggage to identify any traces of 
explosive material. 

In January 2010, TSA revised explosives detection requirements for the 
EDS (hereinafter referred to as "2010 EDS requirements") to better 
address current threats. The specific details included in the 2010 EDS 
requirements such as the physical characteristics and minimum masses 
of each of the explosives types that EDS machines must detect are 
classified. As highlighted in TSA's 2010 EBSP acquisition strategy, to 
improve its existing checked-baggage-screening capability, TSA plans to 
procure 260 EDSs (that is, the current acquisition) capable of meeting the 
2010 EDS requirements. In addition to acquiring new EDSs to better 
address current threats, TSA is planning to procure new EDSs because 
many of the currently deployed machines are nearing the end of their 
expected service lives. 

You requested that we review TSA's efforts to enhance explosives 
detection requirements for checked-baggage-screening technologies and 
to ensure that newly acquired and currently deployed explosives 
detection technologies meet the enhanced requirements. Specifically, this 
report assesses (1 ) the extent to which TSA has revised explosives 
detection requirements and deployed EDSs and ETDs to meet these 
revised requirements; (2) any challenges that TSA and S&T have 
experienced in implementing the EDS acquisition; and (3) the extent to 
which TSA's approach to its EDS acquisition meets best practices for 
schedule and cost estimates and includes plans for potential upgrades to 
deployed EDSs. 

This report is a public version of the prior sensitive report that we 
provided to you in May 201 1. DHS deemed some of the information in the 
prior report as sensitive, which must be protected from public disclosure. 
Therefore, this report omits certain sensitive information about EDS and 
ETD explosives detection requirements, including descriptions of those 
requirements as well as timeframes for their implementation, the number 
of EDSs meeting the requirements, and challenges associated with data 
collection for explosives. This report addresses the same questions as 
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the sensitive report. Also, the overall methodology used for both reports is 
the same. 

To determine the status of TSA's efforts to revise explosives detection 
requirements for checked baggage screening, we reviewed TSA's 
explosives detection requirements for EDSs established in January 2010 
and compared them to explosives detection requirements previously 
established in 2005 and 1998 to determine how the requirements differed. 
We also identified, analyzed, and discussed with TSA officials how the 
2010 EDS requirements compare with current explosives detection 
requirements for the ETD established in 2006. We also visited three of 
the Department of Energy's national laboratories — Lawrence Livermore 
National Laboratory, Los Alamos National Laboratory, and Sandia 
National Laboratories — to determine the status of additional efforts to 
further revise the requirements. 

To identify any challenges that TSA is experiencing in implementing the 
EDS acquisition, we reviewed documentation from TSA's EBSP — the 
program responsible for operational testing, procurement, deployment, and 
maintenance of checked-baggage-screening technologies. Among other 
things, we reviewed the 2010 EBSP acquisition strategy, the program's risk 
management plan, the technical specifications for its procurement of new 
EDSs, and DHS acquisition guidance and directives. We conducted 
multiple interviews with EBSP program officials regarding the program's 
approach to the current EDS acquisition and received updates on revisions 
to TSA's planned approach to the acquisition and timelines. We also 
compared TSA's acquisition efforts with internal control standards. 2 In 
addition, we conducted site visits and/or telephone interviews with all six 
EDS vendors competing in the current EDS procurement and also obtained 
their views regarding TSA's approach to the competitive procurement. 
While information we obtained from these interviews may not be 
generalized across the industry as a whole, we were able to obtain the 
perspectives of all companies planning to compete for the current EDS 
procurement, and they were able to provide an understanding of their 
companies' abilities to develop EDSs that meet the 2010 EDS 
requirements. 



2 GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21 .3.1 
(Washington, D.C.: November 1999). 
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To assess the extent to which TSA's approach for its EDS acquisition 
meets best practices for schedule and cost estimates, and includes plans 
for potential upgrades to deployed EDSs, we assessed the original and 
revised schedule for the current EDS acquisition against relevant best 
practices in our Cost Estimating and Assessment Guide to determine the 
extent to which the schedule reflects key estimating practices that are 
fundamental to having a reliable schedule. 3 We compared TSA's efforts 
with recommended practices that we previously identified for sound 
acquisition planning. 4 We also conducted interviews with DHS officials 
and conducted site visits to S&T's TSL to obtain additional perspective on 
TSA's efforts to deploy EDSs that meet the 2010 requirements. Further, 
we visited Tyndall Air Force Base (AFB), Florida, where the Air Force 
Research Laboratory, TSL, and the Department of Energy's Lawrence 
Livermore National Laboratory are assisting S&T's Explosives Division's 
data collection efforts. Regarding TSA's planning for potential upgrades to 
already deployed EDSs, we interviewed TSA and S&T officials to identify 
the explosives detection technologies that are currently used for checked- 
baggage-screening. We also identified the number of currently deployed 
EDSs that meet the 1998 and 2005 EDS explosives detection 
requirements, the number of currently deployed ETD that meet the 2002 
and 2006 ETD requirements, any challenges involved and expected in 
upgrading EDS detection capabilities, and TSA's plans to upgrade EDSs 
to meet its 2010 requirements. Additionally, in our site visits and 
telephone interviews with the six vendors as we previously discussed, we 
asked vendors to also provide their perspectives on TSA's approach to 
upgrading currently deployed EDSs as well as those to be deployed in the 
future. 

We conducted this performance audit from September 2009 through May 
201 1 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit to 
obtain sufficient, appropriate evidence to provide a reasonable basis for 
our findings and conclusions based on our audit objectives. During the 
course of our review, we revised the engagement objectives and scope to 
facilitate a broader examination of TSA's efforts to revise its explosives 



3 GAO, GAO Cost Estimating and Assessment Guide, GAO-09-3SP (Washington, D.C.: 
March 2009). 

4 See, for example, GAO, Department of Homeland Security: Assessments of Selected 
Complex Acquisitions, GAO-10-588SP (Washington, D.C.: June 2010). 
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detection requirements and related schedule for the EDS acquisition, a 
revision that increased the time for completing this audit. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. Appendix I contains additional 
information on the objectives, scope, and methodology of our review. 



Background 



The Roles of TSA and S&T After the September 1 1 , 2001 , terrorist attacks, the Aviation and 

Transportation Security Act (ATSA) was enacted. 5 Among other things, 
ATSA required that TSA provide for the screening of all checked baggage 
for explosives transported on flights departing U.S. commercial airports. 6 
Pursuant to ATSA, TSA deployed EDS and ETD equipment to screen 
checked baggage and identify potential threats from explosives. 7 While 
TSA is responsible for operating or overseeing the operation of checked- 
baggage-screening equipment, TSA and S&T share responsibilities for 
the research and development of checked-baggage-screening 
technologies. During fiscal year 2006, most research and development 
functions within DHS, including TSA, were consolidated within S&T. 8 After 
this consolidation, S&T assumed primary responsibility for the research, 
development, and related test and evaluation of airport checked- 
baggage-screening technologies. S&T also assumed responsibility from 
TSA for the TSL, which tests and evaluates technologies under 
development against TSA-established detection requirements. TSA 
continues to be responsible for: identifying the requirements for new 



5 Aviation and Transportation Security Act (ATSA), Pub. L. No. 107-71, 115 Stat. 597 
(2001). 

6 Pub. L. No. 107-71, § 110(b), 115 Stat, at 614-16 (codified as amended at 49 U.S.C. § 
44901). 

7 49 U.S.C. § 44901 (d)-(e) (as amended, requiring that TSA deploy sufficient equipment to 
ensure that all checked baggage be screened using explosives detection equipment (EDS 
and ETD) as soon as practicable but in no event later than December 31 , 2003). 

Consistent with the Homeland Security Act of 2002, DHS undertook to coordinate and 
integrate most of its research, development, demonstration, testing, and evaluation 
activities within S&T. See Pub. L. No. 107-296, § 302(12) 116 Stat. 2135, 2164 (2002). 
Whereas TSA received a separate appropriation for research and development activities 
related to aviation security through fiscal year 2005, beginning in fiscal year 2006, TSA 
research and development activities were rolled into the S&T appropriation. 
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checked-baggage-screening technologies; operationally testing and 
evaluating technologies in airports; and procuring, deploying, and 
maintaining technologies. TSA relies on S&T as the central coordination 
point to manage all work related to explosives that involve the TSL, 
Lawrence Livermore National Laboratory, Los Alamos National 
Laboratory, Sandia National Laboratories, and the Air Force Research 
Laboratory at Tyndall AFB. 



Deploying EDSs and ETDs According to TSA, since fiscal year 2001 , TSA has made over $8 billion 

available to EBSP for activities related to checked-baggage screening. 
TSA uses two types of technology for checked baggage screening — the 
EDS and the ETD — at 462 U. S. commercial airports. EDS is used to 
identify suspicious bulk items or anomalies in checked baggage that 
could be explosives or detonation devices. In airports that have EDS, it is 
used for primary screening of checked baggage while ETD machines are 
used for secondary screening of checked baggage to help resolve EDS 
alarms. Additionally, at airports without EDS, the ETD machines are used 
for primary screening of checked baggage. See figure 1 for a photograph 
of an EDS machine and figure 2 for a photograph of an ETD machine. 



Figure 1: Stand-alone Explosives Detection System (EDS) 




Source: GAO. 
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Figure 2: Explosives Trace Detection (ETD) Machine 




Source: TSA. 



TSA deploys EDSs in multiple configurations, such as an in-line 
configuration and a stand-alone configuration. The in-line configuration 
integrates EDS with an airport's baggage handling system — the conveyor 
system that sorts and transports baggage for loading onto an aircraft. 
EDS in stand-alone configurations are separate baggage screening units 
that are not integrated with a baggage-handling system and are typically 
located in an airport lobby although they may also be located in other 
airport locations. Checked baggage is manually loaded and unloaded on 
stand-alone EDS machines. 

As of October 2010, TSA had 2,297 EDS machines in its fleet, 1,938 of 
which were deployed at airports in the United States. 9 At airports and 
terminals that do not use EDSs, ETD machines are used for primary 
checked-baggage screening. Typically, ETDs are used for primary 
screening of checked baggage at smaller airports. These airports typically 
do not have EDSs for primary screening of checked baggage. As of 
February 201 1 , TSA estimated that there were about 5,200 ETD 
machines used for the primary or secondary screening of checked 
baggage at U.S. commercial airports. 



According to TSA officials, 359 EDS units are currently stored in warehouses. EDS 
vendors deliver new EDS machines to TSA's warehouses where they are prepared for 
deployment to airports. 
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S&T and TSA's TSA certifies the EDS it deploys to commercial airports for screening 

Certification and Testing checked baggage, based on tests performed by the TSL. 10 Specifically, 
Process T ^ A certifies tnat EDSs, alone or as part of an integrated system, can 

detect, under realistic operating conditions, the amounts, configurations, 
and types of explosive material which would be likely to be used to cause 
catastrophic damage to an aircraft, using requirements developed in 
consultation with experts from outside TSA. 11 Furthermore, TSA 
periodically reviews threats to civil aviation security, including: 

• explosive material that presents the most significant threat to civil 
aircraft; 

• the minimum amounts, configurations, and types of explosive material 
that can cause, or would be expected to cause, catastrophic damage 
to aircraft in air transportation; and 

• the amounts, configurations, and types of explosive material that can 
be detected reliably by existing or reasonably anticipated, near-term 
explosive detection technologies. 12 

Currently, TSA requires that EDSs undergo three types of testing — 
certification testing, integration testing, and operational testing — before it 
will purchase such equipment. First, TSA verifies that vendors' explosives 
detection systems meet — that is, are capable of detecting in accordance 
with — the TSA established explosives detection requirements through the 
certification testing process. TSA's decision to certify an EDS relies on 
the results of independent test and evaluation performed at the TSL. Prior 
to certification testing, TSL conducts preliminary evaluations of vendors' 
EDS, known as certification readiness testing (CRT) and pre-certification, 
to determine the extent to which vendors are ready to enter certification 
testing. During CRT, TSL provides feedback to vendors on their EDS's 
strengths and weaknesses in detecting explosives in order to help 
vendors make necessary adjustments to their detection software. 
Second, in addition to being certified that the EDS can meet explosives 
detection requirements, EDSs being deployed in an in-line configuration 



TSA also certifies the ETD that it deploys to commercial airports for screening checked 
baggage; however, this report primarily focuses on the EDS. 

"See 49 U.S.C. § 44913(a)(1). 

12 See 49 U.S.C. § 44912(b). 
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must also undergo integration testing. As part of this testing, machines 
deployed in an in-line configuration must demonstrate in a controlled 
environment that they can be successfully integrated within the baggage- 
handling systems used for checked baggage. Finally, following 
certification and integration testing, EDSs undergo operational testing in 
an airport setting to demonstrate that they can reliably and effectively 
function in a live airport environment. 



TSA Has Revised 
Explosives Detection 
Requirements for 
Checked-Baggage- 
Screening Systems, 
but Faces Challenges 
in Deploying 
Equipment to Meet 
the Requirements 



In 2005, TSA revised explosives detection requirements for the EDS; 
however, some number of the EDSs are currently operating at the levels 
to detect explosives as set forth only in the 2005 requirements. 13 When 
TSA established the 2005 requirements, it did not have a plan that 
identified the appropriate time frames needed to deploy EDSs to meet the 
requirements. In January 2010, TSA again revised the EDS explosives 
detection requirements and plans to deploy EDSs meeting these 
requirements in a tiered and phased approach over a number of years. 
One tier of requirements consists of three levels and expanded the 
number and types of explosives that EDSs must detect. 14 TSA is in the 
process of developing another tier of requirements, which will refine the 
amount (for example, minimum mass) of an explosive that can cause 
catastrophic damage to an aircraft. If TSA deploys EDSs that fully meet 
the one tier of the requirements, TSA must ensure that ETD machines are 
capable of detecting all of the explosives that EDSs will be able to detect 
to minimize any potential screening difference between the EDS and 
ETD. 



TSA Did Not Establish a 
Plan to Ensure That 
Currently Deployed EDSs 
Meet the 2005 
Requirements 



In November 2005, TSA revised its explosives detection requirements for 
EDS that had previously been established in 1998 by the Federal Aviation 
Administration. However, as of January 201 1 , some number of the EDSs 
in TSA's fleet are configured to detect explosives at the levels established 
only in the 2005 requirements. The remaining EDSs are configured to 
detect explosives at 1998 levels. When TSA established the 2005 



13 The specific details regarding this issue are considered sensitive security information. 

14 For purposes of this report, unlike TSA officials who defined the implementation of EDS 
explosives detection requirements as having occurred at the time that EDSs are certified, 
we define implementation as having occurred at the time that EDSs are certified, 
purchased, and deployed to airports nationwide. 
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requirements, it did not have a plan with the appropriate time frames 
needed to deploy EDSs to meet the requirements. Standard practices for 
program and project management state that specific desired outcomes or 
results should be conceptualized, defined, and documented in the 
planning process as part of a road map, along with the appropriate steps, 
time frames, and milestones needed to achieve those results. 

Despite the absence of a plan, TSA officials stated that they must conduct 
testing to compare the false alarm rates for machines operating at one 
level of requirements to those operating at another level of 
requirements. 15 According to TSA officials, the results of this testing would 
allow them to determine if additional staff are needed at airports to help 
resolve false alarms once the EDSs are configured to operate at a certain 
level of requirements. 16 TSA officials reported that they had anticipated 
this operational testing was to be completed in March 201 1. 

According to agency officials, TSA did not begin this operational testing 
immediately after the previous explosives detection requirements were 
established in November 2005 because the agency officials were aware 
at the time of a potential further revision of the requirements based upon 
a planned computer modeling effort to revise the detection standards that 
became known as Project Newton. TSA and S&T officials told us they 
had planned to use the results from Project Newton to further revise the 
explosives detection requirements to reflect the mass of an explosive that 
would cause catastrophic damage to an aircraft. Although Project Newton 
did not begin until 2007, TSA officials told us that they were aware of 
plans to further revise the requirements prior to the initiation of Project 
Newton and delayed operational testing in anticipation of the results of 
the computer modeling effort. As of April 201 1, the EDS explosives 
detection requirements have not been changed based on results of the 
computer modeling because Project Newton is still underway, though 
TSA officials told us that they plan to use the results of Project Newton to 



15 A false alarm is when the system alarms even though a threat is not present. A false 
alarm rate is defined as the percentage of times that a false alarm occurs in a given 
number of trials. In addition to a false alarm rate, there are other metrics for system 
performance, such as the detection rate. 

16 TSA officials told us that although this testing was not completed prior to pursuing 
procurements for previous acquisitions, they plan to complete this testing as a part of the 
current EDS acquisition. 
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later define a tier of the 2010 EDS requirements. (We discuss the status 
of Project Newton in more detail in app. II.) 

However, once it became apparent that the results of Project Newton 
would not become available to further revise the requirements, TSA did 
not establish a plan with time frames for completing the additional testing 
related to staffing. 17 Standard practices for planning and project 
management suggest TSA should have defined the operational testing 
plan and milestones as part of a road map for assessing potential staffing 
changes when these EDSs were first deployed after the 2005 
requirements were established. Establishing reasonable time frames to 
complete the operational testing could help TSA ensure it achieves its 
desired goal of activating EDSs capable of detecting the explosives 
established in the 2005 requirements in a timely manner. 



TSA Plans to Deploy EDSs 
That Meet the 2010 
Explosives Detection 
Requirements Using a 
Phased Approach 



In January 2010, TSA revised the EDS checked-baggage explosives 
detection requirements partly in response to credible and immediate 
threats to civil aviation. TSA plans to meet the 2010 EDS requirements 
using a tiered and phased approach: one tier is to be implemented over a 
number of years and expands the types of explosives that EDSs must 
detect, and another tier adds the use of the results of the computer- 
modeling effort known as Project Newton. TSA is not certain when this 
tier of requirements will be implemented. TSA plans to deploy EDSs that 
meet one tier of the 201 requirements in a phased approach beginning 
in late fiscal year 201 1 as part of its planned EDS acquisition. Regarding 
the other tier, TSA officials stated that Project Newton is to enhance 
TSA's understanding of explosives effects by simulating hundreds of 
explosives tests using computer modeling to determine the effects 
explosives will have when placed in different locations within an aircraft. 
TSA's and S&T's understanding of how explosives affect aircraft has 
largely been based on data obtained from live-fire explosive tests on 
retired aircraft hulls and other data. Project Newton is jointly managed 
and funded by DHS's S&T and TSA. Through fiscal year 2009, S&T and 
TSA had invested about $12.5 million in national laboratories for 
computer modeling activities as part of Project Newton, according to a 



TSA officials have reported that all EDSs purchased under the 2010 requirements will 
have 2010 detection software installed and activated once the machines are deployed. 
Thus, new machines certified to meet the 2010 requirements will operate at these 
detection levels in airports. 
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senior TSA official. We discuss Project Newton and its budget in more 
detail in appendix II. 

TSA plans to implement one tier of requirements in a phased approach 
that consists of three levels (see fig. 3). 18 As our past work has shown, an 
incremental or phased approach to implementing requirements can 
reduce risk and make a program more achievable by providing more time 
to develop and test key technologies. 19 TSA officials told us that the ability 
to develop EDS is likely to become increasingly complex as the 
implementation of requirements progresses. According to TSA, it expects 
to begin procuring EDSs to meet the 2010 requirements in July 201 1. 
Consequently, if TSA is successful in deploying EDSs that meet all three 
levels of the 2010 requirements, TSA's EDS fleet would be certified to 
detect more explosives than a fleet meeting the 1998 or 2005 
requirements. 



Figure 3: TSA's Plan to Deploy EDSs That Meet One Tier of the 2010 Requirements 
in Phases 



Level C 



Level B 



EDS procurement estimated 
to begin July 201 1 



Requirements to be 
implemented over a 
number of years 



Level A 



Requirements to be 
implemented over a 
number of years 



Source: GAO analysis of TSA data. 



The 2010 EDS explosives detection requirements contain classified information; 
therefore, this report does not discuss the names of the explosives or other specific 
information about the explosives. 

19 See, for example, GAO, Defense Acquisitions: Restructured JTRS Program Reduces 
Risk, but Significant Challenges Remain, GAO-06-955 (Washington, D.C.: September 
2006). 
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Deploying EDSs That Meet 
Subsequent Levels of the 
2010 Requirements Could 
Affect How EDS Alarms 
Are Resolved 



ETD machines are currently certified to detect some different explosives 
than those EDSs that meet the 2005 EDS detection requirements. 
However, if TSA purchases and deploys EDSs that fully meet Level C of 
the 2010 EDS requirements, ETD machines are not required to detect all 
of the explosives that can be detected by EDSs. TSA's existing ETD 
explosives detection requirements identify the types and quantities of 
explosives materials, that is, traces of explosives, that must be detected 
and the minimum detection rate for each category of explosive. 

According to TSA officials, the ETD explosives detection requirements 
have not been revised because TSA wanted to first focus on revising the 
EDS explosives detection requirements in time for its planned EDS 
acquisition, which is aimed at replacing and upgrading its fleet of EDSs 
used to screen checked baggage. TSA officials stated that they are 
developing a combined set of explosives detection requirements that 
could eventually result in the ETD and EDS machines detecting the same 
explosives. TSA officials stated that the combined set of EDS and ETD 
requirements would not be expected to be approved until sometime in 
calendar year 201 1 . Although combined detection requirements for ETD 
and EDS are to help ensure that both machines can detect the same 
explosives, the machines are not expected to be required to detect the 
same amounts of explosives because the purpose of the ETD is to detect 
traces of explosives in nanograms while the EDS is designed to detect 
larger amounts of explosives. 

At all airports that use EDSs to screen checked baggage, ETD machines 
are used in conjunction with EDSs to screen checked baggage for 
explosives. At these airports, if an EDS alarms — indicating that checked 
baggage may contain an explosive or explosive device that cannot be 
cleared — ETD machines are used as a secondary screening device in 
order to attempt to resolve the alarm. However, the differences between 
the EDS and ETD requirements may impact the resolution of EDS alarms 
by the ETD in the future. According to TSA's 2010 EBSP Acquisition 
Strategy, additional equipment — other than the ETDs currently 
deployed — is to be employed to support alarm resolution when EDSs that 
meet the new checked-baggage explosives detection requirements are 
deployed. However, the acquisition strategy does not specify what 
additional equipment or screening protocols will be employed to resolve 
alarms nor does the strategy discuss whether TSA will continue to use 
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ETD equipment to resolve EDS alarms. 20 According to TSA, the agency is 
currently evaluating what additional technologies and/or changes to 
screening protocols may be needed to address any potential gap in 
capability between the newly certified EDSs and ETDs used for alarm 
resolution. If TSA begins operating EDSs that detect explosives in 
subsequent phases of the 2010 requirements, this potential screening 
difference between EDS and ETD will exist until TSA deploys additional 
equipment and/or implements new screening protocols that could be used 
for secondary screening. 

In commenting on this issue, TSA officials stated that checked-baggage- 
screening technologies are only one layer of security and that other layers 
of security exist to help address potential threats to the aviation security 
system. However, officials agreed that they have not yet developed new 
screening protocols or deployed additional equipment that will address 
the potential gap in screening capability between EDS and ETD if the new 
EDSs are deployed. Standards for program management require that 
specific desired outcomes or results be conceptualized, defined, and 
documented in the planning process as part of a road map, along with the 
appropriate steps and time frames needed to achieve those results. 21 
Because TSA decided to revise explosives detection requirements for 
EDSs prior to revising the ETD requirements, the differences in the 
requirements may affect TSA's capability to detect the 2010-required 
levels until TSA identifies technologies or protocols needed to address 
the potential gap. Without a plan to ensure that secondary-screening 
devices or protocols are in place to resolve EDS alarms if EDSs are 
deployed with additional capability, it will be difficult for TSA to provide 
assurances that the potential capability gap has been resolved. 



20 We refer to the screening protocols to identify that they are part of the checked-baggage 
screening process; although we reviewed the protocols, we did not evaluate them 
because we focused on the checked-baggage screening technologies for this report. 

21 Project Management Institute, The Standard for Program Management (2006). 
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TSA Has Faced 
Challenges in 
Implementing Plans 
for the Current EDS 
Acquisition 



TSA has developed an EBSP acquisition strategy to guide its efforts to 
improve its fleet of checked-baggage-screening machines, but has faced 
several challenges in implementing plans for the current EDS acquisition 
under this strategy. First, TSA has experienced challenges in collecting 
explosives data on the physical and chemical properties of certain 
explosives needed by vendors to develop EDS detection software and 
needed by TSA before procuring and deploying EDSs to meet the 2010 
requirements. TSA and S&T have experienced these challenges because 
of problems associated with safely handling and consistently formulating 
some explosives. Second, the challenges related to data collection for 
certain explosives have resulted in problems carrying out the EDS 
procurement as planned. Specifically, attempting to collect data for 
certain explosives while simultaneously pursuing the EDS procurement 
has delayed the EDS acquisition schedule by at least 7 months. Finally, 
EDS vendors have expressed concerns about the extent to which TSA is 
communicating with the business community about the current EDS 
procurement. 



TSA's EBSP Acquisition 
Strategy Is Being Used to 
Guide Efforts to Improve 
the Explosive Detection 
Capabilities of the EDS 



In July 2010, DHS approved TSA's current acquisition strategy for the 
EDS, and under this strategy, TSA plans to increase the threat detection 
capabilities of the EDS using a competitive procurement to purchase and 
deploy EDS beginning in 201 1 . According to TSA officials, most of the 
previous EDS acquisitions were sole source procurements. 22 However, 
TSA is implementing a competitive procurement for the current EDS 
acquisition to, in part, meet the EBSP acquisition strategy's goals and 
objectives. 23,24 Furthermore, the EBSP acquisition strategy calls for 
acquiring new EDSs as part of the recapitalization plan to replace aging 
EDS. Under the current EDS procurement, TSA plans to award contracts 



The Federal Acquisition Regulation defines a "sole source acquisition" as "a contract for 
the purchase of supplies or services that is entered into or proposed to be entered into by 
an agency after soliciting and negotiating with only one source." See 48 C.F.R. § 2.101(b). 
For purposes of this report, the terms "acquisition" and "procurement" are 
interchangeable. 

23 See H.R. Conf. Rep. No. 111-298 (accompanying H.R. 2892, Department of Homeland 
Security Appropriations Act, 2010) (Oct. 13, 2009) at 78 (providing that TSA shall move to 
a fully competitive EDS procurement process no later than September 30, 2010). 

24 The acquisition strategy's objectives also include meeting the EBSP mission needs by 
minimizing life-cycle costs, minimizing impact to airport operations, and providing a flexible 
security infrastructure capable of meeting future airline traffic and industry needs. 
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to purchase 260 EDSs, including those in its recapitalization plan, at an 
estimated cost of $256 million during the fourth quarter of fiscal year 
201 1. 25 Until TSA begins purchasing machines under the current EDS 
acquisition to meet 2010 requirements, the agency has continued to 
purchase EDSs under existing contracts with current vendors. 



TSA Faces Challenges in 
Collecting Explosives Data 
Needed to Procure EDSs 
That Meet the 2010 
Requirements 



TSA and S&T have experienced a number of challenges related to 
collecting data on some explosives data needed to procure and deploy 
EDSs that meet the 2010 requirements. These data are needed both by 
vendors to develop EDS detection software and by the TSL for the 
certification testing process and includes such information as the physical 
and chemical properties of explosives. Participants in the data collection 
effort include S&T; the TSL, which is taking the lead; the Air Force 
Research Laboratory (AFRL) at Tyndall AFB, Florida; and Lawrence 
Livermore National Laboratory. The AFRL is assisting the TSL because the 
AFRL facility at Tyndall AFB, Florida, is better equipped to safely handle 
certain explosives as part of the testing and data collection efforts than the 
TSL facility, according to S&T and Air Force officials. 26 



In the course of collecting data, TSL and AFRL officials determined that 
some of the explosives were very unstable or volatile and special care 
and procedures were required to reliably and safely handle them. This 
caused delays as TSA and S&T were unable to provide vendors with all 
of the data on the explosives or simulants, so that vendors could create 
and test the software used to detect them. 27 TSL and AFRL also collected 
scans of explosives as another means to provide vendors with needed 
data. Because micro-computed technology (micro-CT) data are provided 
by scanning smaller, less dangerous, amounts of each explosive, TSL 
and AFRL officials did not face the same challenges in safely 
synthesizing and determining the physical and chemical properties of the 
micro-CT data that they faced in working with larger amounts, known as 



25 TSA's Passenger Screening Program acquires ETD machines on behalf of the EBSP for 
use in checked-baggage screening. 

26 The AFRL at Tyndall Air Force Base is supporting S&T, TSL, and TSA in developing 
explosives data through fiscal year 2012. S&T is in the process of identifying other 
facilities that can safely test explosives. 

27 A simulant is a "safe to handle" material designed to appear to EDS technology as a 
real explosive. 
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full threat weight, of explosives. TSL officials told us that they provided 
the micro-CT data to help vendors in developing their explosives 
detection software. Specifically, TSA was able to distribute the micro-CT 
data to vendors in early fiscal year 201 0, and five of the six vendors we 
interviewed stated that these data were of limited use in developing their 
explosives detection software. For example, one vendor stated that the 
micro-CT data provided some guidance, but that there were too many 
unknowns to fully use the data to develop their explosives detection 
software. Further, TSL officials stated that providing the micro-CT data to 
vendors only served as an intermediary step to providing full threat-weight 
data that vendors needed to develop their explosives detection software. 
TSL officials stated that the micro-CT data could not provide vendors with 
all of the data they needed to fully develop their explosives detection 
software to meet the 2010 EDS requirements because vendors need 
scans of the full threat weight of explosives on their respective EDSs to 
finish developing their detection software. 

Because of the limitations of simulants and micro-CT data, TSA and S&T 
decided to collect and distribute scans of the explosives to vendors using 
the full threat weight of the explosives specified in the 2010 EDS 
requirements. These scans were collected using each vendor's 
respective EDS equipment. 28 TSA has distributed some, but not all, of the 
full threat-weight data needed by vendors to develop EDS detection 
software. TSL officials stated that they needed full threat-weight data to 
conduct certification testing. Additionally, five of the six vendors we 
interviewed agreed that the full threat-weight data will be necessary in 
order for vendors to develop their explosives detection software. 
However, all six vendors noted that because of concerns about the safety 
of handling certain explosives, they are relying on TSA for the full threat- 
weight data. Further, four of the vendors said that in the past they had 
access to some explosives and could collect their own data to develop 
and test their detection software in order to prepare for certification 
testing. However, because the vendors cannot safely handle certain 
explosives, they are reliant upon the data provided by TSA. 
Consequently, until S&T completes the data collection on all identified 
explosives being performed at TSL and the AFRL facility at Tyndall AFB, 



The EDS uses a computed tomography X-ray source that rotates around a bag, 
obtaining a large number of cross-sectional images that are integrated by a computer 
which automatically triggers an alarm when objects with the characteristic of explosives 
are detected. 
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TSA cannot provide all of the data that vendors need to develop their 
explosives detection software and prepare for certification, nor can the 
TSL start certification testing of new equipment as part of the current EDS 
acquisition. 



Challenges Related to 
Collecting Explosives Data 
Have Delayed the EDS 
Acquisition Schedule 



TSA's plans to award contracts for the current EDS acquisition have been 
delayed by at least 7 months, in part, due to the challenges experienced 
by S&T related to collecting explosives data. TSA officials stated that, 
initially, they planned to conduct the current EDS acquisition separately 
from efforts to collect the data needed to deploy EDSs that meet the 2010 
requirements. Specifically, TSA officials stated that they planned to 
complete the data collection before initiating the procurement to buy 
EDSs that meet the 2010 requirements. However, officials stated that 
they subsequently decided to collect explosives data at the same time as 
implementing the current EDS acquisition because TSA and other 
stakeholders believed that the data collection effort would be 
straightforward and that the new requirements could be easily applied to 
machines procured in the current EDS acquisition. Additionally, program 
officials stated that procuring and deploying EDSs that meet the 2010 
requirements in a phased approach (that is, implementing Level C first, 
then Level B, then Level A) would help to mitigate any additional 
challenges and some of the risks associated with collecting data needed 
for the 201 requirements. 



However, TSA and S&T officials acknowledged that pursuing the 
competitive procurement and explosives data collection at the same time 
had been more challenging than originally anticipated and had presented 
problems for the current EDS acquisition. TSA officials stated that all of 
the 260 EDSs they plan to purchase in 201 1 will be upgraded to meet all 
of the 2010 EDS requirements at a later date. 



In our prior reports regarding acquisitions, we reported on the elevated 
risk of poor program outcomes from the substantial overlap of 
development, test, and production activities. Specifically, we have 
identified development cost increases, additional delays in manufacturing 
and testing schedules, and increased financial risk due to pursuing 
procurement before testing is complete. 29 By separating the effort to 



See, for example, GAO, Joint Strike Fighter: Significant Challenges Remain as DOD 
Restructures Program, GAO-10-520T (Washington, D.C.: March 11, 2010). 
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collect data on explosives needed to meet the new requirements from the 
related competitive procurement, TSA and S&T would have more time to 
collect data identifying the physical and chemical properties of explosives, 
provide vendors with the time needed to develop detection software, and 
attempt to pass CRT and certification testing without the added pressure 
of an acquisition deadline. For example, by completing data collection for 
each of the phases of the 2010 EDS requirements prior to pursuing 
procurements for EDSs that meet those requirements, TSA could avoid 
additional delays to the acquisition schedule due to any data collection 
challenges. To help avoid these challenges in the future, TSA officials 
stated that they do not plan for subsequent procurements of EDS capable 
of meeting the more stringent explosives detection requirements until 
after the data collection for these explosives has been fully completed. 

We recognize that it is difficult in such situations to identify firm 
milestones. However, TSA has not documented its revised approach for 
conducting the needed data collection and related procurements 
sequentially rather than simultaneously. TSA does not yet have a 
documented strategy in place for deploying EDSs beyond July 201 1; such 
a strategy would be valuable because TSA plans to complete the 
implementation of all of the requirements at an undetermined time after 
July 201 1 . Standard practices for program management state that the 
successful execution of any plan includes identifying in the planning 
process the schedule that establishes the timeline for delivering the 
plan. 30 Documenting a plan to separate data collection efforts and 
certification from future procurements could help TSA ensure it avoids the 
challenges it has encountered during the current procurement. 



EDS Vendors Report 
Concerns about the Extent 
to Which TSA is 
Communicating Effectively 
about the Current 
Procurement 



Officials from five of six EDS vendors we interviewed expressed concerns 
about the extent to which TSA has communicated effectively with vendors 
interested in the current procurement. Specifically, these five vendors 
expressed concerns about the timeliness in which TSA responded to their 
questions regarding the current procurement or the manner in which TSA 
communicated important schedule changes, or both. Standards for 
Internal Control in the Federal Government state that management 
should ensure there are adequate means of communicating with and 



30 Project Management Institute, The Standard for Program Management © (Second 
Edition, 2008). 



Page 19 



GAO-1 1-740 Aviation Security 



obtaining information from external stakeholders that may have a 
significant impact on the agency achieving its goals. Additionally, the 
Federal Acquisition Regulation (FAR) encourages exchanges of 
information among all interested parties, from the earliest identification of 
a requirement through the receipt of the proposal. The FAR further states 
that the purpose of exchanging information is to improve the 
understanding of government requirements and industry capabilities, 
thereby allowing vendors to judge whether or how they can satisfy the 
government's requirements. The improved understanding resulting from 
such information exchange also enhances the government's ability to 
obtain quality supplies and services at reasonable prices and, among 
other things, potentially increases efficiency in vendors' proposal 
preparations. However, five out of six vendors we interviewed said TSA 
often did not provide information or respond to their questions in a timely 
manner, if at all. 31 For example, four out of these five vendors said TSA 
did not answer their questions in a timely manner, in one case taking 
several months to provide answers to questions posted via a question 
tracker accessible online to all interested vendors. Meanwhile four of the 
five vendors' officials stated TSA did not respond at all to some of their 
questions, while officials from the fifth vendor stated they were frustrated 
with how long it took TSA to answer their questions. 

Officials from two vendors stated that the lack of timely communication 
regarding schedule changes for the EDS acquisition caused them to incur 
additional costs allocating extra resources and time to meet the original 
deadline. Specifically, officials from one vendor noted that they spent 
additional costs on personnel to aggressively pursue software development 
for the planned start of certification readiness testing (CRT), despite not 
having all of the full threat-weight explosives data TSA had intended to 
provide. Subsequently, these officials told us, TSA did not announce to 
vendors that CRT would be delayed until one week prior to the original 
deadline. EBSP officials stated that, because vendors had not yet received 
all of the full threat weight explosives data, they should have been aware 
that CRT was not going to happen according to the established schedule. 
However, EBSP officials agreed that providing vendors with a revised 
schedule prior to the previously established deadline would have helped 



31 Officials from the sixth vendor did not comment on the extent to which TSA responded to 
questions in a timely manner. 
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promote greater vendor understanding about the proposed changes to 
TSA's acquisition strategy. 

TSA stated that it has taken a number of important steps to alleviate 
confusion and provide as much information to the vendors as possible. 
Among other things, at the start of the current procurement, TSA 
conducted three conferences with industry, called "industry days," to 
provide a forum for sharing information with the vendor community 
regarding the current EDS acquisition. TSA also reported sharing multiple 
draft versions of the requirements documents and soliciting vendor 
comments. Additionally, TSA officials stated that they shared draft copies 
of the detection requirements and held individual classified meetings 
during the industry days with each interested vendor to obtain input 
regarding the acquisition. Finally, TSA stated that it also allowed vendors 
to use government owned equipment and paid for engineering services 
associated with the testing to help offset vendor costs. Although EBSP 
officials stated that they have made a concerted effort to be responsive to 
vendors' questions and to call vendors directly when issues such as 
schedule changes arose, EBSP officials agreed that the agency did not 
always effectively communicate with vendors in a timely manner. 
Establishing a process for more timely communication with vendors 
competing for the current EDS procurement could help TSA to ensure 
that vendors have all of the information they need to meet TSA's needs 
for new checked baggage screening equipment. 



Improved Acquisition 
Planning Could Help 
TSA Avoid Further 
Delays and Potential 
Cost Overruns for the 
EDS Procurement 



TSA does not have an integrated master schedule (IMS) for the EBSP, 
and TSA's schedule for the current EDS acquisition, which is only a part 
of the program, does not fully meet best practices for preparing an 
acquisition schedule. 32 Additionally, while TSA completed an initial cost 
estimate for the EBSP, TSA officials reported that the current cost 
estimate does not reflect the anticipated costs of purchasing EDSs to 
meet the 2010 EDS requirements. To meet the explosives detection 
requirements established in January 2010, TSA plans to upgrade the 
detection software of a currently unknown number of the deployed EDSs 
and 260 of the EDSs to be purchased under the current acquisition after 



TSA established the EBSP as a long-term program to procure, test, deploy, and maintain 
checked-baggage-screening equipment; it is expected to reach full capacity in 2019. 
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they are deployed to airports. However, TSA has not yet developed a 
plan or cost estimate for the planned upgrades. 



TSA Does Not Have a 
Schedule for the EBSP and 
Has Not Established a 
Reliable Schedule for the 
Current EDS Acquisition 

EBSP As part of EBSP's responsibility to provide equipment to screen all 

checked baggage originating at U.S. commercial airports, it is acquiring 
and deploying explosives detection technology to replace aging systems 
and meet emerging threats. While TSA established the EBSP as a long- 
term program to procure, test, deploy, and maintain checked-baggage- 
screening equipment, TSA officials confirmed in December 2010 that 
there is currently no IMS for the EBSP. Among other things, best 
practices and related federal guidance call for a program schedule to be 
programwide in scope, meaning that it should include the integrated 
breakdown of the work to be performed by both the government and its 
contractors over the expected life of the program. 33 Without an IMS 
identifying long-term plans for the EBSP, it is difficult for TSA to have a 
comprehensive program view of the work that must be completed to 
deliver explosive detection technology to replace aging systems and meet 
emerging threats. Without such a view, a sound basis does not exist for 
knowing with any degree of confidence when and how the program will be 
completed. 

EDS Acquisition While there is no IMS for the EBSP, TSA has established a schedule for 

the current EDS acquisition. However, while the schedule identifies 
activities through the first contract award — scheduled for July 201 1 — of 
the current EDS procurement, our analysis shows that it does not identify 
activities planned for subsequent award windows. 34 Additionally, TSA has 



See, for example, GAO-09-3SP; and OMB, Capital Programming Guide V 2.0, 
Supplement to Office of Management and Budget Circular A-1 1, Part 7: Planning, 
Budgeting, and Acquisition of Capital Assets (Washington, D.C.: June 2006). 

34 TSA refers to the first planned EDS contract award as Window 1 and subsequent 
contract awards as Windows 2 and 3, respectively. However, TSA has not yet established 
contract award dates for Windows 2 and 3 in the schedule. 
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encountered a number of challenges in implementing the schedule. For 
example, according to TSA officials, TSA originally planned to award the 
first EDS contract in December 2010 in order to procure machines 
required to meet Level C of the 2010 EDS requirements. However, TSA 
has since revised the schedule due to the challenges of collecting 
explosives data needed before development of EDS can be completed 
and certification testing of the machines can begin. 35 

Based on the revised schedule, certification testing began in late 2010, 
according to TSA, so that in July 201 1 the first EDS contract can be 
awarded to procure machines that meet one part of the Level C 
explosives detection requirements. Furthermore, while TSA has stated 
that it plans to procure and deploy 640 additional EDSs at an estimated 
cost of approximately $964 million during fiscal years 2012 through 2015, 
it is unclear when TSA plans for those machines to meet the remaining 
2010 EDS requirements. 

As of March 201 1 , TSA officials estimate that it will take a number of 
years to certify EDSs that meet all three levels — C, B, and A — of the 2010 
requirements. However, the officials stated that they cannot fully develop 
these plans until they can evaluate the capability of the equipment to 
meet these requirements. This is expected to happen during the testing 
process associated with the current EDS procurement. TSA officials 
stated that they plan to deploy EDSs that meet the full set of Level C, B, 
and A requirements, but more precise planning, including establishing 
timelines, cannot occur until TSA better understands the potential for the 
EDS equipment to meet those requirements. However, best practices 
state that a comprehensive schedule should at least reflect all activities 
planned for a project even though some activities may be tentative and 
there may be uncertainties in schedule estimates due to, among other 
things, limited data. 36 

In addition to the challenges TSA has encountered in carrying out the 
schedule as originally planned, based on our analysis, the current schedule 



We assessed TSA's initial schedule, which was provided to GAO in July 2010; met with 
TSA officials to discuss our assessment; and provided officials with suggestions for 
corrective action. Subsequently, TSA revised the schedule and provided it to GAO in 
October 201 0. We completed a separate assessment of TSA's revised schedule. 

36 GAO Cost Estimating and Assessment Guide, GAO-09-3SP (Washington, D.C.: 
March 2009). 
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leading up to the first contract award is not reliable. 37 Best practices state 
that the success of a large-scale system acquisition, such as the current 
EDS acquisition, depends in part on having a reliable schedule that 
identifies: 

• when the program's set of work activities and milestone events will 
occur, 

• how long they will take, and 

• how they are related to one another. 

Best practices also call for the schedule to expressly identify and define 
the relationships and dependencies among work elements and the 
constraints affecting the start and completion of work elements. 
Additionally, best practices indicate that a well-defined schedule also 
helps to identify the amount of human capital and fiscal resources that are 
needed to execute an acquisition. However, based on our assessment of 
both the original as well as an updated version of the schedule, TSA's 
schedule for the current EDS acquisition does not fully comply with nine 
best practices for preparing a schedule as shown in table 1. Appendix III 
has additional information about GAO's assessment of the extent to 
which TSA's schedule meets each best practice. 



37 GAO-09-3SP. 
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Table 1: Implementation of Best Practices in Electronic-Baggage-Screening 
Program Schedule as of December 2010 



Best practice 


Explanation of best practice 


Degree of 
implementation 


1. 


Capturing all activities 


Defining in detail the work to be 
completed, including activities to 
be performed. 


(5 


o 
z. 


Sequencing all activities 


Listing activities in the order in 
which they are to be carried out. 


3 


3. 


Assigning resources to 
all activities 


Identifying the resources needed 
to complete the activities. 


3 


4. 


Establishing the 
duration of all activities 


Determining how long each activity 
will take to execute. 


3 


5. 


Integrating all activities 
horizontally and 
vertically 


Achieving aggregated products or 
outcomes by ensuring that 
products and outcomes associated 
with other sequenced activities are 
arranged in the right order, and 
dates for supporting tasks and 
subtasks are aligned. 


3 


6. 


Establishing the critical 
path for all activities 


Identifying the path in the schedule 
with the longest duration through 
the sequenced list of key activities. 




7. 


Identifying reasonable 
float between activities 


Determining the amount of time 
that a predecessor activity can slip 
before the delay affects successor 
activities. 




8. 


Conducting a schedule 
risk analysis 


Using statistical techniques to 
predict the level of confidence in 
meeting a project's completion 
date. 


o 


9. 


Updating the schedule 
using logic and 
durations to determine 
the dates for all 
activities 


Continuously updating the 
schedule to determine realistic 
start and completion dates for 
program activities based on 
current information. 





Source: GAO analysis of TSA information. 

Note: We intended to assess the EBSP schedule based on the nine best practices, but when we 
identified that TSA did not have an IMS for the EBSP, we assessed the EDS acquisition schedule. 



• The program provided evidence that satisfies the entire criterion (fully met). 

* The program provided evidence that satisfies about half of the criterion (partially met). 

© The program provided evidence that satisfied less than half of the criterion (minimally met), 
o The program did not provide evidence that satisfies any of the criterion (not met). 



Page 25 



GAO-1 1-740 Aviation Security 



Although TSA's schedule does not fully comply with any of the nine best 
practices, TSA has taken action to partially or minimally meet eight of the 
best practices. For example, consistent with best practice 4, the schedule 
establishes the duration of all activities and properly reflects how long 
each activity should take. However, while the schedule establishes the 
duration of all activities, 61 percent of activities represented in the 
schedule are based on a 7-day calendar that does not account for 
holidays. Similarly, our analysis found that, consistent with best practice 
5, the schedule is vertically integrated; however, issues with sequencing 
logic in the schedule prevent it from being fully horizontally integrated. 
Vertical and horizontal integration ensures that products and outcomes 
associated with other sequenced activities are arranged in the right order 
and that dates for supporting tasks and subtasks are aligned. Other areas 
of the schedule that remain unaddressed also reflect weaknesses that 
limit its usefulness as a program management tool. For example, the 
schedule does not fully identify the resources needed to do the work or 
the availability of these resources. Specifically, the schedule does not 
reflect what labor, material, and overhead are needed to complete key 
activities for the program. Resource information would assist the program 
office in forecasting the likelihood of activities being completed based on 
their projected end dates. If the current schedule does not allow for 
insight into current or projected over-allocation of resources, then the risk 
of the program slipping is significantly increased. 

Additionally, TSA officials did not complete a schedule risk analysis when 
developing the schedule. A schedule risk analysis may be used to 
determine the level of uncertainty and to help identify and mitigate the 
associated risks. In the absence of a schedule risk analysis, the 
acquisition faces the risk of delays to the scheduled completion date if 
any delays were to occur on critical path activities. Furthermore, without 
this information, TSA is limited in its ability to answer questions such as 
how likely it is to complete the project on time and which risks are most 
likely to delay the project. Similarly, without a valid critical path, EBSP 
management lacks a clear picture of the tasks that must be performed to 
achieve the acquisition's target completion date. While TSA officials noted 
that they had no staff or expertise to complete a schedule risk analysis, 
TSA provided no explanation as to why a schedule consistent with the 
other eight best practices had not been developed. TSA officials stated 
that the EDS acquisition is one of the largest acquisition programs in 
DHS. However, the absence of a reliable schedule makes it difficult for 
management to predict with any degree of confidence whether the 
estimated completion date for the acquisition is realistic. Furthermore, 
without the development of a schedule that meets scheduling best 
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practices, TSA is limited in its ability to monitor and oversee the progress 
of the billions of dollars being invested in the procurement of new EDSs. 



The EBSP's Current Life- 
Cycle Cost Estimate Does 
Not Reflect Anticipated 
Costs for Purchasing EDSs 
That Meet the 2010 EDS 
Requirements 



The EBSP does not yet have an up-to-date approved life-cycle cost 
estimate in place, and as a result, DHS has no reliable basis for 
understanding how much the program will cost. While TSA reported that it 
had completed a life-cycle cost estimate (LCCE) for the EBSP in May 
2010, program officials reported in February 2011 that the estimate is 
currently being revised to reflect assumptions related to the current EDS 
acquisition. 38 Specifically, officials indicated that the May 2010 LCCE did 
not include the anticipated costs for purchasing any EDSs that meet the 
revised 2010 requirements. TSA officials stated that they are working to 
revise the LCCE to reflect the anticipated costs of the current EDS 
acquisition and expected to complete the LCCE by the end of April 201 1. 
Additionally, after conducting a review of the LCCE that was completed in 
May 2010, DHS's Cost Analysis Division (CAD) found that the LCCE 
needed more comprehensive data and that its accuracy could not be 
determined. 39 As a result, the DHS Acquisition Review Board directed the 
CAD to develop an appropriate cost estimate, including a reconciliation 
with the EBSP's LCCE. 40 In January 201 1 , officials in DHS's CAD stated 
that they had initiated work on the independent cost estimate for the 
EBSP but were only able to complete the portion of the estimate related 
to current detection capabilities in the Level C requirements for one tier. 
Officials stated that the lack of detail in program requirements for some of 
Level C and all of Levels B and A limited their ability to develop an 
estimate that would be usable for budgetary purposes. CAD officials 
further noted that significant portions of the total EBSP program have yet 
to be defined and estimated. During the course of our review, the 



38 Life-cycle costs include all resources and associated cost elements required to develop, 
produce, deploy, and sustain a particular program from initial concept through operations, 
support, and disposal. Acquisition costs include costs for all supplies and services for a 
designated investment. 

39 According to DHS, the Director, Cost Analysis Division, located in the Office of the Chief 
Procurement Officer, serves as the focal point within DHS for cost analysis and estimating 
policy, process, and procedure. 

40 The DHS Acquisition Review Board is responsible for reviewing major DHS acquisitions 
for executable business strategy, resources, management, accountability, and alignment 
to strategic initiatives. According to DHS, the EBSP is either at or very near the top of the 
Acquisition Review Board's priority list. 
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anticipated completion date of the LCCE has been delayed multiple times 
and was expected to be completed at the end of April 201 1 . As a result, 
we were unable to evaluate TSA's approach to developing the cost 
estimates for the program. 

We reported in June 2010 that inaccurate or incomplete cost estimates 
were often a factor in cost growth for DHS programs we previously 
reviewed. We also reported that initial cost estimates for most DHS 
programs were often developed after the start of acquisition activities, so 
they do not capture earlier cost changes. 41 Further, our best practices for 
cost estimation state that estimates are integral to determining and 
communicating a realistic view of likely cost and schedule outcomes that 
can be used to support a program including planning the work necessary to 
develop, produce, and install equipment. However, because TSA had not 
established a cost estimate that accurately reflects the anticipated costs of 
the acquisition prior to initiating the current EDS procurement, it is unclear 
how DHS could determine if the budget for the EBSP is reasonable. 
Furthermore, in the absence of an approved cost estimate and baseline 
financial information for the current EDS acquisition, including the costs of 
purchasing machines that meet the 2010 EDS requirements, TSA has 
limited information to make essential cost-informed program decisions. 

Although we were unable to evaluate TSA's cost estimates for the 
program, the fact that TSA's schedule for the EDS acquisition does not 
meet best practices for schedule estimating also raises questions about the 
credibility of the program's LCCE. For example, the absence of a schedule 
risk analysis would have made it difficult for officials to account for the cost 
effects of schedule slippage when developing the LCCE. Best practices for 
cost estimation state that because some program costs such as labor, 
supervision, rented equipment, and facilities cost more if the program takes 
longer, a reliable schedule can contribute to an understanding of the cost 
impact if the program does not finish on time. 42 The program's success 
depends on the quality of its schedule and an integrated schedule is key to 
managing program performance and is necessary for determining what 
work remains and the expected cost to complete the work. In a memo from 
the DHS Under Secretary for Management dated July 10, 2008, DHS 



GAO, Department of Homeland Security: Assessments of Selected Complex 
Acquisitions, GAO-10-588SP (Washington, D.C.: June 2010). 

42 See GAO-09-3SP. 
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endorsed the use of best practices that we identified and stated that DHS 
would be utilizing them as a "best practices" approach in the future. 43 
However, in the absence of a reliable schedule to guide cost estimates, 
having a current cost estimate that reflects anticipated costs for the EDS 
acquisition, or submitting the revised LCCE to DHS for departmental 
approval, it is unclear how TSA utilized a best practices approach in 
developing cost estimates for the program. 



TSA officials stated that they expect to upgrade an unknown number of 
the current fleet of 2,297 EDSs and 260 of the EDSs to be purchased 
under the current acquisition after they are deployed to airports to fully 
meet all phases of the 2010 requirements. 44 However, similar to when 
TSA revised the EDS explosives detection requirements in 2005, it has 
no plan in place outlining how it will approach these upgrades. 
Specifically, TSA has not established an upgrade plan or conducted an 
analysis to determine what type of approach to upgrading deployed EDSs 
is likely to be most feasible, efficient, or effective. TSA officials stated that 
there are too many unknowns at this time regarding potential approaches 
to upgrading the fleet of EDSs. Standards for program management 
require that specific desired outcomes or results be conceptualized, 
defined, and documented in the planning process as part of a road map, 
along with the appropriate steps and time frames needed to achieve 
those results. 45 Until TSA develops a plan identifying how it will approach 
the upgrades for currently deployed EDSs — and the plan includes such 
items as estimated costs, the number of machines that can be upgraded, 
and the number of times a given machine must be upgraded to meet the 
2010 EDS requirements — it will be difficult for TSA to provide reasonable 
assurance that its upgrade approach is feasible or cost-effective. 

TSA's 2010 acquisition strategy identifies the planned purchase of 900 
EDSs over the next 5 years, but it does not indicate what level 
requirements those EDSs will be required to meet. For the currently 
deployed equipment that will not be replaced, TSA would need to 



DHS endorsed the use of best practices outlined in GAO's 2007 Cost Estimating Guide 
(GAO-07-1134SP). 

44 According to TSA officials, while the current fleet of EDSs totals 2,297 machines, 1 ,938 
of the machines are currently deployed in airports. 

45 Project Management Institute, The Standard for Program Management (2006). 



TSA Has No Plan in Place 
Outlining How It Will 
Upgrade Deployed EDSs to 
Fully Meet the 2010 
Requirements 
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upgrade the equipment to meet the 2010 requirements. TSA officials 
stated that they may not upgrade all of the current fleet of EDSs to the 
2010 EDS requirements because in some cases, certain models of the 
EDS may not be upgradeable and in other cases, it may ultimately be 
more cost effective to replace older EDSs with new machines. According 
to TSA, upgrading EDSs will require an assessment of currently deployed 
EDS' detection capabilities and that the results of the assessment will 
affect the EDS program's schedule, budget, and detection goals. TSA 
was working with a consulting firm to modify a computer model that will 
be used to project the costs of the upgrades if TSA were to use a time- 
phased installation for the upgrades. While TSA officials were working 
with a consulting firm, they have not yet established a plan for how they 
will approach the upgrades. TSA officials further stated that the number of 
upgrades TSA performs on currently deployed equipment will depend on 
the cost of the upgrades, the level of complexity of the upgrades, and 
whether the upgrades can be conducted in the airports or must be 
performed in the factory. 

TSA's approach to deploy EDSs that meet the 2010 requirements could 
result in the same EDSs being upgraded multiple times in order to first 
meet all of the Level C requirements and to then meet the Levels B and A 
explosives detection requirements. 46 For example, TSA's decision to 
revise its acquisition strategy and deploy EDSs that meet the Level C 
requirements in a phased approach could result in upgrading the same 
currently deployed machines twice before they may have to be upgraded 
a third time to meet Level B requirements and then upgraded a fourth 
time to meet Level A requirements. Moreover, based on TSA's schedule 
for the current EDS acquisition, by the time some or all of the 260 new 
EDSs under the current EDS acquisition have been deployed in airports, 
TSA may have approved a subsequent tier of the EDS explosives 
detection requirements, which could involve upgrading the machines 
again or replacing these newly purchased and deployed machines 
because they cannot meet the subsequent tier of explosives detection 
requirements. Therefore, TSA may procure and deploy 260 EDSs that will 
only be used in airports for a short period of time before they will need to 
be upgraded, possibly multiple times, or replaced with new machines. 



TSA officials stated that, if they purchase EDSs that meet a part of the Level C 
requirements, they plan to upgrade those machines to meet other parts of the 
requirements. Once EDSs are purchased to meet all parts of the Level C requirements, 
the machines will need to be upgraded to meet Levels B and A requirements. 
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TSA officials told us that they will evaluate the need to upgrade EDSs to a 
subsequent tier at the time those requirements are finalized. 

TSA officials stated that they initially delayed the analysis of the upgrade 
approach until the 2010 EDS explosives detection requirements were 
approved, an approval that occurred in January 2010. TSA officials 
subsequently stated that their plan to upgrade deployed EDSs is included 
in the recapitalization strategy due to be completed at the end of May 
201 1 . According to TSA, vendors that have previously sold EDSs to TSA 
are to be asked to also include proposals to upgrade their currently 
deployed machines when submitting proposals for the current EDS 
procurement. Specifically, vendors are to be asked to include a plan for 
upgrading their currently deployed EDS equipment as well as cost 
estimates for the upgrades. TSA plans to then analyze the feasibility and 
costs of the vendors' proposals. However, TSA officials stated that the 
equipment upgrades may or may not be implemented as part of the 
contract award and that TSA has discretion regarding which aspects of 
the contracts to implement. According to TSA, the total number of EDSs 
to be upgraded and the associated costs will not be known until the 
agency receives proposed upgrade plans and cost estimates from EDS 
vendors in summer 201 1 . According to TSA officials, any upgrades are 
not to occur until calendar year 2012 at the earliest and will depend on 
available funding and complexity of the upgrades. 

TSA officials as well as officials from three of six current EDS vendors told 
us that they are confident that currently-deployed EDSs can be upgraded 
to meet Level C requirements. 47 Specifically, TSA officials stated that the 
EDS vendors can rewrite the detection software to provide the capability to 
detect the 2010 EDS requirements. Minor hardware changes, such as new 
computer chips, are also expected to be made as part of these upgrades 
according to the TSA officials. The officials stated that, after they approve 
the software and hardware upgrades, EDS vendors will install the upgrades 
on the machines in the airports. Officials from three EDS vendors noted 
that they believe upgrades to the EDSs can be made in the airports when 
regularly-scheduled routine maintenance work is conducted. 



Officials from another vendor stated that upgrades could be made to the EDSs in 
airports, but did not specify when the upgrades could be made. Officials from the other 
two EDS vendors do not have EDSs currently deployed in airports and, therefore, did not 
comment on the feasibility of upgrades. 
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Once deployed EDSs have been upgraded to fully meet the Level C 
requirements, TSA will have to make decisions about how to ensure 
these machines can meet subsequent phases of the 2010 EDS 
requirements (Levels B and A). Officials from all six EDS vendors stated 
that given the absence of additional data on the explosives that will be 
included in subsequent phases of the 2010 EDS requirements, it is 
difficult to know precisely what must be done to upgrade newly-purchased 
equipment. Therefore, none of the officials from the six vendors could 
provide estimates for the cost to upgrade EDSs to meet all of the 
requirements for one tier. However, officials from two vendors estimated 
the cost to upgrade new EDSs to meet Level C requirements at $50,000 
to $150,000 per machine. 48 An official from one vendor stated that the CT 
technology currently used in EDSs might not be sufficient to detect Level 
A requirements and that an as yet undeveloped technology may be 
needed. The official noted that this could result in substantially higher 
costs to upgrade the current fleet of machines to Level A requirements 
than it would cost to upgrade machines to Level B requirements. 
Similarly, officials from two other vendors stated that meeting Level A 
requirements may require either new technology or a combination of 
current technologies instead of only using an EDS. 

Although TSA and vendor officials expressed confidence that deployed 
EDSs can be upgraded, TSA officials also confirmed that the agency has 
never previously upgraded the detection software of deployed EDS or 
ETD machines to meet revised explosives detection requirements. 
Additionally, even though TSA has estimated that it will take a number of 
years to certify new EDSs to fully meet Levels B and A of the 2010 
requirements, TSA has not yet developed similar time frames to upgrade 
deployed equipment. Given the number of unknowns associated with 
upgrading EDSs, it is unclear how long it will take the agency to upgrade 
deployed EDSs to meet Levels C, B, and A of the 2010 requirements. 
Furthermore, TSA has identified the EDS upgrade effort as a high 
program risk. Consequently, TSA and vendor officials' confidence that it 
will be feasible and cost effective to upgrade deployed machines at 
airports may be unwarranted as it has not been based on experience, 
supported by analysis, or a documented plan. 



48 The four other EDS vendors did not have cost estimates for upgrades to existing 
machines. 
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TSA faces a complex task in its efforts to address explosives threats in its 
current and future procurements and existing fleet of checked-baggage- 
screening systems. The complexity of this task is amplified when taking 
into account the large volume of checked baggage that TSA must screen 
for explosives without disrupting commerce. TSA's plan to procure and 
deploy EDSs that meet the 2010 requirements in a phased approach that 
spans a number of years is aimed at allowing more time to collect 
necessary explosives data, test key technologies, and provide a means 
for TSA to continue to purchase EDSs to meet its needs for new checked- 
baggage-screening equipment at the nation's commercial airports. 
However, TSA officials recognized that if TSA deploys EDS capable of 
detecting all explosives included in the 2010 EDS requirements, TSA 
must ensure that ETD machines are capable of detecting all of the 
explosives that EDSs will be able to detect to minimize any potential 
screening difference between the EDS and ETD. Without a plan to help 
ensure that additional screening devices or protocols are in place to 
resolve EDS alarms if EDSs are deployed that detect a broader set of 
explosives than existing ETD machines used to resolve EDS screening 
alarms, it will be difficult for TSA to provide reasonable assurance that a 
potential capability gap has been resolved. 

By separating the effort to collect data needed to meet the 2010 EDS 
requirements from the related competitive procurement, TSA would have 
more time to identify the physical and chemical properties of the 
explosives, collect full threat weight data, provide vendors with the time 
needed to develop detection software, and attempt to pass CRT and 
certification testing without the added pressure of an acquisition deadline. 
TSA also faces additional challenges related to the agency's plans for 
implementing the current EDS procurement. For example, the lack of 
timely communication with vendors may impact vendors' abilities to 
ensure they can meet TSA's needs for the current EDS acquisition. By 
establishing a process to communicate with vendors in a timely manner, 
TSA could help ensure that vendors have the information necessary to 
meet TSA's needs for new checked-baggage-screening equipment. 
Moreover, by addressing challenges related to planning for the 
acquisition, TSA may be able to better avoid further delays and potential 
cost overruns for the current procurement. Specifically, completing a 
reliable IMS that fully meets the nine best practices could help DHS and 
TSA management to predict whether the estimated acquisition completion 
date is realistic and manage program performance. Once a reliable 
schedule is in place, TSA can in turn revise current cost estimates for the 
program to better reflect actual acquisition costs including, for example, 
the potential cost impacts resulting from schedule slippage to give 
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program decision-makers a more accurate and comprehensive view of 
current and projected program costs. 

As TSA plans to deploy EDSs that meet the 2010 requirements, it is 
critical that TSA plans its approach to ensure that all airports with EDS 
equipment are capable of detecting the required explosives. Because 
TSA has not yet upgraded most of the deployed EDSs to meet certain 
requirements, many EDSs are only capable of detecting certain 
explosives. Moreover, of the EDSs currently deployed, TSA is currently 
operating some number of them at the capability needed to detect the 
explosives identified in the 2005 requirements, although activating the 
software and operationally testing the machines to detect the 2005 
requirements would help address this issue. 

As part of TSA's phased approach to meet the 2010 EDS requirements, 
TSA may have to upgrade many of its currently deployed EDSs and 
hundreds of newly purchased EDSs over a period of years, upgrades that 
may require significant investments in new technologies to help meet 
more stringent explosives detection requirements. However, until TSA 
develops a plan identifying how it will approach the upgrades for currently 
deployed EDSs — and the plan includes such items as estimated costs, 
the number of machines that can be upgraded, time frames for upgrading 
them, and the number of times a given machine must be upgraded to 
meet the 2010 EDS requirements — it will be difficult for TSA to provide 
reasonable assurance that its upgrade approach is feasible or cost- 
effective. 



Recommendations for ^° ' ne '' 3 ensure tnat TSA takes a comprehensive and cost-effective 

approach to the procurement and deployment of EDSs that meet the 

Executive Action 2010 EDS requirements and any subsequent revisions, we recommend 

that the Assistant Secretary for TSA take the following six actions: 

• Develop a plan to ensure that screening devices or protocols are in 
place to resolve EDS alarms if EDSs are deployed that detect a 
broader set of explosives than existing ETD machines used to resolve 
EDS screening alarms. 

• Develop a plan to ensure that TSA has the explosives data needed for 
each of the planned phases of the 2010 EDS requirements before 
starting the procurement process for new EDSs or upgrades included 
in each applicable phase. 
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• Establish a process to communicate information to EDS vendors in a 
timely manner regarding TSA's EDS acquisition, including information 
such as changes to the schedule. 

• Develop and maintain an integrated master schedule for the entire 
Electronic Baggage Screening Program in accordance with the nine 
best practices identified by GAO for preparing a schedule. 

• Ensure that key elements of the program's final cost estimate reflect 
critical issues, such as the potential cost impacts resulting from 
schedule slippage identified once an integrated master schedule for 
the Electronic Baggage Screening Program has been developed in 
accordance with the nine best practices identified by GAO for 
preparing a schedule. 

• Develop a plan to deploy EDSs that meet the most recent EDS 
explosives-detection requirements and ensure that new machines, as 
well as machines deployed in airports, will be operated at the levels 
established in those requirements. This plan should include the 
estimated costs for new machines and upgrading deployed machines, 
and the time frames for procuring and deploying new machines and 
upgrading deployed machines. 



Agency Comments 
and Our Evaluation 



We provided a draft of this report to DHS on June 23, 201 1 , for review 
and comment. On July 6, 2011, DHS provided written comments, which 
are presented in appendix IV. We also provided relevant excerpts of our 
draft report to DOD and the Department of Energy for review and 
comment. In commenting on our report, DHS stated that it agreed with 
our six recommendations and identified actions planned or underway to 
implement them. DOD provided written technical comments and the 
Department of Energy provided technical comments in an e-mail. Both 
stated that the draft report excerpts related to their respective agencies 
contained accurate information. 



Overall, DHS stated that, because of the urgent need to meet ongoing 
requirements, TSA began addressing many of the issues identified by this 
audit while the audit was being conducted. However, as DHS noted in its 
letter, TSA still needs to complete many actions to resolve the issues 
identified in this report. Additionally, TSA stated that it suspended the 
implementation of the 2005 requirements because of the computer 
modeling effort known as "Project Newton" and then issued the 2010 
detection standards when Project Newton did not yield timely results. 
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Thus, in its comments, TSA confirmed that it is using some number of 
EDSs that meet requirements established in 1998 by the FAA, as we 
reported, an approach that raises questions about how well some of its 
deployed equipment detects current explosives threats. 

In addition, DHS stated that TSA is currently taking steps to collect the 
operational data necessary to support the upgrade of deployed 
equipment based on 2010 detection standards and that the operational 
data-collection effort is to be completed in 201 1 . However, as discussed 
in the report, this could be a difficult endeavor as TSA is still in the 
process of conducting operational testing to determine the staffing 
implications of operating EDSs that meet 2005 explosives detection 
requirements. Therefore, it will have taken TSA 6 years from the time that 
the 2005 EDS explosives detection requirements were issued until this 
operational testing is to be completed. Furthermore, if the results of the 
operational testing show that operating EDS machines, to meet the 2005 
requirements, will require additional TSA staff and/or slow down the rate 
of checked-baggage screening, TSA may have to make difficult decisions 
and trade-offs that could affect aviation security and commerce, and also 
affect the schedule for meeting the 2010 requirements. 

DHS concurred with our first recommendation to develop a plan to ensure 
that screening devices or protocols are in place to resolve EDS alarms if 
EDSs are deployed that detect a broader set of explosives than existing 
ETD machines used to resolve EDS screening alarms. DHS stated that 
TSA convened a working group to assess capability gaps for secondary 
screening technology, evaluate current technology capabilities against the 
capabilities of future EDSs, and prepare a plan to procure any additional 
technology required to ensure alarms can be resolved. DHS expects this 
plan to be finalized by the fourth quarter of fiscal year 2012. While these 
actions and planned actions represent positive steps to fully implement 
the recommendation, TSA should develop a plan to ensure that screening 
devices or protocols are in place to resolve EDS alarms if EDSs are 
deployed that detect a broader set of explosives than existing ETD 
machines used to resolve EDS screening alarms. 

DHS concurred with our second recommendation to develop a plan to 
ensure that TSA has the data needed for each of the planned phases of 
the 2010 EDS requirements before starting the procurement process for 
new EDSs or upgrades included in each applicable phase. DHS 
commented that TSA modified its strategy for the EDS's competitive 
procurement in July 2010 in response to the challenges in working with 
the explosives for data collection and alerted the vendor community on 
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September 3, 2010. DHS stated that the new baseline schedule removed 
data collection from the acquisition process. Additionally, DHS stated that 
TSA is working with DHS S&T to establish a laboratory by summer 201 1 
to support further data collection and independent test and evaluation. 
Although these actions respond in part to the intent of our 
recommendation, separating data collection from the acquisition process 
does not necessarily ensure that the needed data will be available before 
starting the procurement process for the new EDSs or upgrading currently 
deployed EDSs. Consequently, we continue to believe that, to fully 
address our recommendation, a plan is needed to establish a process for 
ensuring that data are available before starting the procurement process 
for new EDSs or upgrades for each applicable phase. Developing and 
following such a plan would assist TSA in implementing the acquisition 
and making upgrades in an efficient and effective manner and would 
benefit DHS in its oversight role of TSA by allowing DHS to determine 
progress against the plan. 

DHS concurred with our third recommendation to establish a process to 
communicate information to EDS vendors in a timely manner regarding 
TSA's EDS acquisition, including information such as changes to the 
schedule. In the letter, DHS stated that TSA has a process for 
communicating information to the vendor community and will continue to 
follow this process in adherence with guidelines outlined in the Federal 
Acquisition Regulation. DHS also stated that TSA significantly changed 
the business model to procure checked-baggage-screening equipment 
from what has historically been a sole-source environment to a 
competitive environment, resulting in significant improvements in 
communication with industry. In addition, according to DHS, TSA has 
already made a number of efforts to improve the quality and frequency of 
communication with industry, but TSA recognizes the complexity 
associated with many of the acquisitions currently ongoing. As such, TSA 
acknowledged that there are opportunities to continue to improve 
communication with the vendor community and will take steps to ensure 
that vendors are provided with the most current information possible in an 
efficient manner. Since the agency did not provide us with evidence of 
how it plans to ensure more timely and effective communications with 
vendors in the future, we continue to believe that such a process is 
needed to ensure that TSA officials are aware of the specific guidelines to 
follow to communicate with vendors about current and future acquisitions. 
Our meetings with vendors indicated that TSA's communications with 
them continue to leave room for improvement. 
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DHS concurred with our fourth recommendation to develop and maintain 
an IMS for the entire EBSP in accordance with the nine best practices 
identified by GAO for preparing a schedule. DHS commented that TSA 
has already begun working with key stakeholders to develop and define 
requirements for an IMS and to ensure that the schedule aligns with the 
best practices outlined by GAO. DHS stated that this effort is expected to 
be completed by the second quarter of fiscal year 2012. In addition, DHS 
stated that, as the program matures and increases its focus on flexible 
and upgradeable technology, an IMS will ensure close coordination 
among the program's procurement, deployment, recapitalization, and 
upgrade capabilities, and that the EBSP IMS will be updated as a result of 
these efforts to be in accordance with the nine best practices. While these 
actions and planned actions are steps toward implementing our 
recommendation, to fully implement the recommendation, TSA needs to 
develop and maintain an IMS for the entire EBSP in accordance with the 
nine best practices identified by GAO for preparing a schedule. 

DHS concurred with our fifth recommendation to ensure that key elements 
of the program's final cost estimate reflect critical issues, such as the 
potential cost impacts resulting from schedule slippage. Such a slippage 
might be identified once an IMS for the EBSP has been developed in 
accordance with the nine best practices identified by GAO for preparing a 
schedule. DHS stated that TSA is working to update the EBSP LCCE to 
incorporate cost estimates associated with enhanced detection, work that 
should be completed in the fourth quarter of fiscal year 201 1 . DHS also 
stated that, per the recommendations of GAO and DHS, TSA is developing 
a master schedule to document timelines associated with various projects. 
DHS further stated that risks to the costs and schedules will be analyzed 
and that the risk analysis will produce confidence intervals for the life-cycle 
costs to the program. Although TSA discussed activities to address the 
EBSP LCCE, to fully implement this recommendation, it will be important 
that key elements of the program's final cost estimate reflect critical issues, 
such as the potential cost impacts resulting from schedule slippage 
identified once an IMS for the EBSP has been developed in accordance 
with the nine best practices. 

DHS concurred with our sixth recommendation to develop a plan to 
deploy EDSs that meet the most recent EDS explosives detection 
requirements and ensure that new machines, as well as machines 
deployed in airports, will be operated at the levels established in those 
requirements. This plan should include the estimated costs for new 
machines and upgrading deployed machines, and the time frames for 
procuring and deploying new machines and upgrading deployed 
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machines. DHS commented that TSA has a plan in place to evaluate and 
implement the most recent certified algorithms on the existing fleet of 
deployed EDSs, assuming the evaluation results in minimal to no 
operational impact. In contrast, our recommendation calls for a plan to 
deploy new EDSs as well as to upgrade existing EDSs in airports to meet 
the 2010 EDS explosives detection requirements and, importantly, ensure 
that new machines will be operated at the levels established in those 
requirements. As we discussed in the report, some number of the EDSs 
in airports are operating at a level that meets the 2005 explosives 
detection requirements. Our recommendation is intended to ensure that 
TSA operates all EDSs in airports to meet the most recent requirements, 
which are currently the 2010 requirements. Consequently, we continue to 
believe that a plan is needed describing the approach that TSA will use to 
deploy EDSs that meet the most recent EDS explosives detection 
requirements and ensure that new machines, as well as machines 
deployed in airports, will be operated at the levels established in those 
requirements. 

TSA also provided written technical comments, which we incorporated in 
the report, as appropriate. 



As agreed with your office, unless you publicly announce the contents of 
this report earlier, we plan no further distribution until 10 days from the 
report date. At that time, we will send copies of this report to the 
Secretary of Homeland Security, the Assistant Secretary of the 
Transportation Security Administration, and appropriate congressional 
committees. This report also will be available at no charge on the GAO 
Web site at http://www.gao.gov. 

If you or your staffs have any questions about this report, please contact 
me at (202) 512-8777 or LordS@gao.gov. Contact points for our Offices 
of Congressional Relations and Public Affairs may be found on the last 
page of this report. GAO staff who made major contributions to this report 
are listed in appendix V. 




Stephen M. Lord 
Director 

Homeland Security and Justice 
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Appendix I: Objectives, Scope, and 
Methodology 



This report discusses: (1 ) the extent to which TSA has revised explosives 
detection requirements and deployed EDSs and ETDs to meet these revised 
requirements; (2) any challenges that TSA and the Department of Homeland 
Security's (DHS) Science and Technology Directorate (S&T) have 
experienced in implementing the EDS acquisition; and (3) the extent to which 
TSAs approach to its EDS acquisition meets best practices for schedule and 
cost estimates, and includes plans for potential upgrades to deployed EDSs. 

To determine the extent to which TSA has revised explosives detection 
requirements for checked baggage screening, we reviewed TSA's EDS 
explosives detection requirements for checked baggage screening and 
assessed the extent to which the 2010 detection requirements differed from 
the 2005 detection requirements for EDSs. We compared specific 
explosives 1998, 2005, and 2010 detection requirements to identify 
commercial and homemade variants of the explosives. We also identified, 
analyzed, and discussed with TSA and S&T officials the differences 
between the tiers and multiple levels of explosives detection requirements 
in the 2010 EDS explosives detection requirements. We discussed with 
TSA officials the 2002 explosives detection requirements for the ETD and 
reviewed the 2006 explosives detection requirements for ETD. We also 
compared the 2010 EDS requirements with the 2006 ETD requirements 
and discussed with TSA and S&T officials the differences between the 
explosives detection requirements for the EDS and ETD. We discussed 
TSA's Standard Operating Procedures for resolving EDS and ETD alarms 
with TSA officials. Finally, we visited three of the Department of Energy's 
national laboratories, Lawrence Livermore National Laboratory, Los 
Alamos National Laboratory, and Sandia National Laboratories, to 
determine the status of Project Newton. (For more on Project Newton, see 
app. II.) 

To identify any challenges that TSA has experienced implementing the 
EDS acquisition, we reviewed documentation from TSA's Electronic 
Baggage Screening Program (EBSP), the program responsible 
operational testing, procurement, deployment, and maintenance of 
checked-baggage-screening technologies. Among other things, we 
reviewed available program documentation on the status of its EDS 
acquisition including EBSP strategic plans from previous years as well as 
the most recent EBSP strategy approved in July 2010. We also reviewed 
documentation from the program's first Acquisition Review Board review, 
the EBSP risk management plan, the most recent procurement 
specifications for the EDS, information posted by EBSP for interested 
vendors on FedBizOpps.gov, and DHS acquisition guidance and 
directives. We also interviewed EBSP program officials, including the 
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Appendix I: Objectives, Scope, and 
Methodology 



EBSP program manager, regarding the program's approach to the current 
EDS acquisition and received updates on revisions to the program's EDS 
acquisition strategy and timelines for the current procurement. 

To further understand the challenges TSA and S&T face in preparing for 
new EDSs to meet revised detection requirements, we reviewed 
documentation provided by TSA outlining the agency's plan for deploying 
EDSs that meet the 2010 requirements as well as documentation regarding 
S&T's approach to testing and certification carried out at the Transportation 
Security Laboratory (TSL). We also conducted interviews with TSA and 
S&T officials and conducted site visits to the TSL in Atlantic City, New 
Jersey, and to the Air Force Research Lab (AFRL) facility at Tyndall Air 
Force Base (AFB), Florida, to obtain information on efforts to certify EDSs 
that meet the 2010 requirements. We visited the TSL because that is 
where S&T tests and evaluates transportation technologies including 
checked baggage screening technologies. We visited AFRL because they 
are assisting TSL in their efforts to collect data regarding the physical and 
chemical properties of explosives included in the 2010 EDS requirements 
in preparation to certify EDSs for the current procurement. Additionally, we 
conducted site visits and/or telephone interviews with all six EDS vendors 
competing in the first phase of the current EDS procurement. These 
vendors were able to provide us with an understanding of their companies' 
views regarding TSA's approach to the current procurement as well as 
potential challenges they believe vendors face in preparing to compete for 
the current EDS procurement. While information we obtained from these 
interviews may not be generalized across the industry as a whole, we were 
able to obtain the perspectives of all companies planning to compete for 
the current EDS procurement, and they were able to provide an 
understanding of their companies' abilities to develop EDSs that meet the 
2010 requirements. We also reviewed TSA documentation to identify the 
explosives detection technologies that are used for checked baggage 
screening. Additionally, we interviewed TSA and S&T officials to identify the 
number of currently-deployed explosives detection machines that meet the 
previous and most recent detection requirements, and found the data for 
the number of machines to be sufficiently reliable. 

To determine the extent to which TSA's approach to its EDS acquisition 
meets best practices for schedule and cost estimates and includes plans 
for potential upgrades to deployed EDSs, we determined the extent to 
which TSA had established an integrated master schedule (IMS) for the 
EBSP, and due to the lack of an IMS, assessed the EDS acquisition 
schedule against nine best practices in our Cost Estimating and 
Assessment Guide. We conducted this assessment to determine the extent 
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Methodology 



to which the schedule reflects key estimating practices that are 
fundamental to having and maintaining a reliable schedule. In doing so, we 
independently assessed the schedule for the current EDS acquisition and 
its underlying activities against our nine best practices, as provided to us in 
July 2010. We subsequently interviewed cognizant program officials to 
discuss their use of best practices in creating the schedule and to discuss 
the findings resulting from our review of the schedule. After TSA revised 
the schedule to reflect changes in some of the timelines and provided it to 
us in October 2010, we reviewed the updated schedule and compared it to 
information in the original schedule in order to understand how the new 
schedule was constructed and to determine to what extent TSA had 
resolved weaknesses that we identified in its original schedule. We also 
assessed the schedule against relevant best practices in our Cost 
Estimating and Assessment Guide to determine the extent to which it 
reflects key estimating practices that are fundamental to having a reliable 
schedule. We compared TSA's efforts with internal control standards and 
recommended practices we previously identified for sound acquisition 
planning. 

To further evaluate TSA's planning for the current EDS acquisition, we 
also interviewed TSA, S&T, and EDS vendors' officials to identify any 
challenges involved and expected in upgrading EDS detection 
capabilities, and TSA's plans to upgrade equipment to meet future 
implementations of the 2010 EDS requirements. Also, during our site 
visits and telephone interviews with the six vendors, as discussed 
previously, vendors provided their perspectives on TSA's approach to 
upgrade currently deployed EDSs as well as those to be deployed in the 
future. We also obtained from the six vendors their perspectives on how 
upgrades to deployed EDSs might be accomplished and potential costs 
involved in performing the upgrades. 

We conducted this performance audit from September 2009 through May 
201 1 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit to 
obtain sufficient, appropriate evidence to provide a reasonable basis for 
our findings and conclusions based on our audit objectives. During the 
course of our review, we revised the engagement objectives and scope to 
facilitate a broader examination of TSA's efforts to revise its explosives 
detection requirements and related schedule for the EDS acquisition, 
which increased the time for completing this audit. We believe that the 
evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 
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Appendix II: Department of Homeland Security 
Efforts to Conduct Computer Modeling to 
Establish a Tier of the Explosives Detection 
Requirements for the Explosives Detection System 

The Department of Homeland Security (DHS) Science and Technology 
Directorate (S&T) and Transportation Security Administration (TSA) 
began a project in 2007, known as Project Newton, to identify the 
minimum mass of explosives that could cause catastrophic damage to an 
aircraft. Through 2009, S&T and TSA had invested approximately $12.5 
million in Project Newton modeling activities, according to a senior TSA 
official. A different senior TSA official stated that TSA allocated an 
additional $2.5 million to $3.1 million for Project Newton as of August 
201 0: $1 .0 million to $1 .6 million for incremental development of 
computer models and $1 .5 million to develop a plan to validate the 
models. 1 As part of the effort to understand the effects of explosives 
detonations on aircraft, S&T and TSA have been working to simulate the 
complex dynamics of explosive blast effects on an in-flight aircraft by 
using computer models at three Department of Energy national 
laboratories — Lawrence Livermore National Laboratory, Los Alamos 
National Laboratory, and Sandia National Laboratories. According to TSA 
officials, the current understanding of the effects of explosives on aircraft 
has been largely based on data from live-fire explosives tests conducted 
with retired aircraft hulls at ground level. These officials stated that, 
compared to running a computer simulation, live-fire tests can be more 
expensive, which limits the number of live-fire tests conducted and, 
therefore, the amount of data available for analysis. S&T, TSA, and 
national laboratory officials stated that computer modeling can cost- 
effectively simulate the effects of explosives detonations in various 
locations of different types of aircraft at ground level, which can provide 
significant data for analysis. 

In January 2010, TSA revised the explosives detection requirements for 
the explosives detection system (EDS) and established tiers of explosives 
that are required to be detected. One tier of requirements is to be 
implemented over a number of years. TSA plans to incorporate the 
computer-modeling results into the requirements for a subsequent tier. 
Although TSA expected that the computer-modeling results would be 
used to revise EDS explosives detection requirements as early as 2012, 
as of December 2010, TSA officials were uncertain when the computer- 
modeling results will be used for this purpose because the computer 
models had not been validated. In 2009, TSA established a Blue Ribbon 



1 Computer model validation includes steps to ensure that a model sufficiently simulates 
the actual system. 
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Efforts to Conduct Computer Modeling to 
Establish a Tier of the Explosives Detection 
Requirements for the Explosives Detection 
System 



Panel to, among other things, assess the three national laboratories' 
computer models and their results and comment on whether they were 
valid to be used to revise explosives detection requirements. The panel 
members included DHS and TSA officials as well as officials from 
academia and the private sector. In March 2010, the Blue Ribbon Panel 
recommended that, among other things, before the computer modeling 
results are used to revise EDS explosives detection requirements, the 
computer models and their results should be validated, according to a 
senior TSA official. This official stated that the panel also recommended 
specific locations to add to the computer models, so that the models can 
simulate the effects of explosives detonations in those additional locations 
on the aircraft. Validating the computer models and their results is 
essential before relying on them to revise explosives detection 
requirements. A senior TSA official stated that it will take a number of 
months to validate the computer models, validation that is expected to be 
completed later in 201 1 . 
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Appendix III: GAO's Assessment of the 
Extent to Which the Electronic Baggage- 
Screening-Program Schedule Meets 
Established Best Practices 

In determining the extent to which the Transportation Security 
Administration's (TSA's) Electronic Baggage Screening Program (EBSP) 
schedule meets established best practices, we identified that TSA did not 
have an integrated master schedule (IMS) for the program. As a result, 
we assessed the explosives detection system (EDS) acquisition schedule 
against each of nine best practices. Specifically, we assessed TSA's 
initial schedule, which was provided to GAO in July 201 0, and met with 
TSA officials to discuss our assessment and provided officials with 
suggestions for corrective action. TSA later revised the schedule and 
provided GAO with an updated version in October 2010. We completed a 
separate assessment of TSA's revised schedule. The following table 
presents our two assessments. 



Table 2: Assessment of the Extent to Which the Electronic Baggage-Screening-Program Schedule Meets Best Practices 



Best practice Explanation 



Criterion met 
(July 2010) 



Criterion met 
(October 2010) 



GAO analysis 



1. Capturing all 
activities 



The schedule should 
reflect all activities as 
defined in the 
project's work 
breakdown structure, 
which defines in 
detail the work 
necessary to 
accomplish a 
project's objectives, 
including activities to 
be performed by both 
the owner and 
contractors. 



Minimally met Minimally met 



Initial analysis: TSA officials reported that there is 
currently no IMS for the EBSP and that the initial EDS 
acquisition schedule provided to GAO in July 2010 
only reflects government effort associated with the 
pending contract award for procurement of EDS. The 
EDS acquisition schedule also contains key activities 
performed by other DHS components involved in the 
acquisition, such as TSA, the Department of 
Homeland Security (DHS) Science and Technology 
Directorate's (S&T) Explosives Division (EXD), the 
Transportation Security Laboratory (TSL), the TSA 
Systems Integration Facility (TSIF), Operational 
Testing and Evaluation (OT&E), and Air Force 
Research Lab at Tyndall Air Force Base. The 
schedule includes pre-solicitation and solicitation 
effort for three planned EDS contract awards. 3 

Our analysis found that none of the 901 tasks within 
the schedule are mapped to program work breakdown 
structure (WBS). EBSP officials were not aware of 
how a program WBS would align to either a schedule 
WBS or a WBS being used by DHS Cost Analysis 
Division for an ongoing Independent Cost Estimate. A 
WBS is a valuable communication tool because it 
provides a clear picture of what needs to be 
accomplished and how the work will be done. 
Accordingly, it is an essential element for identifying 
activities in a program's integrated master schedule. 

EBSP officials stated that activities beyond the third 
contract award are not scheduled. While separate 
airport schedules exist that govern the deployment of 
machines purchased under the current procurement 
contract, those schedules are not integrated with the 
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Appendix III: GAO's Assessment of the Extent 
to Which the Electronic Baggage- Screening- 
Program Schedule Meets Established Best 
Practices 



Criterion met Criterion met 
Best practice Explanation (July 2010) (October 2010) GAO analysis 

EDS acquisition schedule because the effort is 
handled under a separate office within TSA. However, 
without an integrated master schedule that accounts 
for all planned government and contractor effort, 
management is unable to reliably estimate planned 
dates beyond the current schedule's end date of 
December 19, 201 1 . Management also has no 
information of the effects on other phases of the 
program — for example, production, deployment, and 
maintenance activities of equipment — if the third 
contract award is delayed. 

Updated analysis: The revised schedule provided to 
GAO in October 2010 represents the new approach to 
the EDS acquisition, with Level C now divided into 
sub-phases. However, the schedule only includes 
activities through the first contract award to purchase 
EDSs to meet part of Level C of the 201 EDS 
requirements under the first contact award. The 
schedule does not include planned contract awards 
for Windows 2 or 3 nor does it reflect when EDSs will 
be required to meet all of Levels C, B, or A. In 
addition, the schedule includes only government 
effort, does not link to any external schedules, and 
does not map activities to a program WBS. 

Initial analysis: Our analysis of the initial EDS 
acquisition schedule provided to GAO in July 2010 
shows that 146 of the 502 remaining activities, or 29 
percent, have missing logic — that is, these activities 
are missing necessary predecessors or successors 
which in turn reduces the credibility of the calculated 
dates. If an activity that has no logical successor slips, 
the schedule will not reflect the effect on the critical 
path, float, or scheduled start dates of downstream 
activities. In addition, we found 21 remaining activities, 
or 4 percent, have "dangling" logic — that is, these 
activities are missing successors off their finish date. 
In other words, the activities could continue 
indefinitely and not affect the start or finish dates of 
downstream activities. 

We found 176 remaining activities (35 percent) with 
Start No Earlier Than constraints. These are 
considered "soft" constraints in that they allow the 
activity to slip into the future based on what happens 
to their predecessor activities. While activities may be 
soft constrained, for example, to represent receipt of 
delivery of equipment, in general, constraining an 
activity's start date prevents managers from 
accomplishing work as soon as possible and 
consumes flexibility in the project. In addition, we 
found 39 Finish No Earlier Than constraints 
(8 percent). These are also considered "soft" date 
constraints because they prevent activities from 



2. Sequencing The schedule should Minimally met Partially met 
all activities be planned so that 
critical project dates 
can be met. Activities 
need to be logically 
sequenced — that is, 
listed in the order in 
which they are to be 
carried out. In 
particular, activities 
that must be 
completed before 
other activities can 
begin (predecessor 
activities), as well as 
activities that cannot 
begin until other 
activities are 
completed (successor 
activities), should be 
identified. This helps 
ensure that 
interdependencies 
among activities that 
collectively lead to the 
accomplishment of 
events or milestones 
can be established 
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to Which the Electronic Baggage- Screening- 
Program Schedule Meets Established Best 
Practices 



Best practice Explanation 



Criterion met Criterion met 

(July 2010) (October 2010) GAO analysis 



and used as a basis 
for guiding work and 
measuring progress. 



finishing earlier than their constraint date. Program 
officials stated they are working to reduce the number 
of constraints within the schedule. They stated that 
many constraints are in place to dictate the start and 
finish dates of testing activities that rely on the 
availability of testing equipment. However, assigning 
calendar-based test equipment resources to testing 
activities eliminates the need for date constraints and 
allows the activities' start and finish dates to respond 
dynamically to changes in the schedule. 

Of the remaining activities, 170 activities (34 percent) 
are linked to their successor activities with lags, 
including 1 1 negative lags. Lags are often used to put 
activities on a specific date or to insert a buffer for 
risk; however, these lags persist even when 
predecessor activities are delayed (that is, when the 
buffer should be consumed). Lags should be justified 
because they cannot have risk or uncertainty. 

Updated analysis: The updated schedule provided to 
GAO in October 2010 has corrected all but two 
missing dependencies on remaining activities (less 
than 1 percent), a marked improvement from the last 
schedule version. The updated schedule also includes 
a lower number of activities with dangling logic: 1 2 
remaining activities (3 percent) are missing 
successors off their finish date. In addition, the new 
schedule includes fewer date constraints than the 
previous version, which improves the dynamic nature 
of the schedule. There are now 13 remaining activities 
with Start No Earlier Than constraints (3 percent), and 
all Finish No Earlier Than constraints have been 
removed. Program officials justified the use of these 
Start No Earlier Than constraints as necessary 
because test units cannot be deployed to each test 
location before January 10, 2011, due to a "black-out" 
period between November 15 and January 10. This 
black-out period, enforced by TSA Security 
Operations, prohibits any changes to airport 
operations other than for emergency purposes. 

However, a large number of lags remain in the 
approved baseline schedule. There are 102 remaining 
activities with lags (27 percent), including 1 1 
remaining activities (3 percent) with leads (negative 
lags). Negative lags are typically discouraged since 
negative time is not demonstrable. 



Assigning 
resources to 
all activities 



The schedule should 
reflect what 
resources (for 
example, labor, 
materials, and 
overhead) are 
needed to do the 



Minimally met Minimally met 



Initial analysis: Program officials stated that beyond 
assigning scheduled activities to integrated product 
teams (IPT), the schedule does not account for 
resources. In addition, our analysis shows 42 (1 1 
percent) of the 369 remaining detail activities have 
IPT assignments. Resource information would assist 
the program office in forecasting the likelihood of 
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to Which the Electronic Baggage- Screening- 
Program Schedule Meets Established Best 
Practices 



Best practice Explanation 



Criterion met 
(July 2010) 



Criterion met 
(October 2010) 



GAO analysis 



work, whether all 
required resources 
will be available when 
needed, and whether 
any funding or time 
constraints exist. 



activities being completed based on their projected 
end dates. If the current schedule does not allow for 
insight into current or projected over-allocation of 
resources, then the risk of the program slipping is 
significantly increased. 

Updated analysis: While the revised schedule 
provided to GAO in October 2010 is not used to 
monitor resources, the revised schedule now includes 
10 resource group names and every remaining activity 
within the schedule is assigned a resource group 
name. The majority of remaining activities are 
assigned to TSIF (229 activities, or 60 percent), and 
another third of the activities are assigned to OT&E 
(121 activities, or 32 percent). The other 8 percent of 
remaining activities are assigned between DHS, TSL, 
the EBSP, and other government agencies. Three 
remaining activities (1 percent) are assigned to the 
EBSP, and represent a total duration of 8 days of 
effort. Program officials stated that while there are 
only three activities assigned to the EBSP, the 
program office has oversight on all scheduled 
activities. However, if there are three activities, or only 
8 days of effort being performed by the EBSP through 
July 201 1 , assuming there are 22 working days per 
month, there are 168 days where the EBSP has no 
scheduled work. As a result, all EBSP effort may not 
be captured in the schedule. 



Establishing 
the duration of 
all activities 



The schedule should 
realistically reflect how 
long each activity will 
take to execute. In 
determining the 
duration of each 
activity, the same 
rationale, historical 
data, and 

assumptions used for 
cost estimating should 
be used. Durations 
should be as short as 
possible and have 
specific start and end 
dates. The schedule 
should be continually 
monitored to 
determine when 
forecasted completion 
dates differ from 
planned dates; this 
information can be 
used to determine 
whether schedule 



Partially met Partially met 



Initial analysis: The majority of remaining activities 
meet best practices for durations. There are 21 (6 
percent) remaining activities with baseline (planned) 
durations longer than 44 days, which exceeds the 
best practice for activity duration. Additionally, no 
activity duration exceeds 84 days, and the two longest 
duration activities represent data collection efforts 
being performed by stakeholders outside of the EBSP 
program office. Representing effort in the schedule 
that is performed by outside organizations is 
considered a best practice because it keeps 
management informed of ongoing work that might 
easily be forgotten until the deliverable is due, and the 
impact on future activities if the deliverable is behind 
schedule. 

Schedule activities are based on one of three work 
calendars: a 7-day workweek, a 5-day workweek, and 
the default 5-day workweek provided with the 
scheduling software. Our analysis found the majority 
of remaining activities (59 percent) are assigned to the 
default calendar, which does not account for holidays. 
In addition, there are 168 activities (33 percent) 
assigned to a 7-day workweek calendar, which also 
does not account for holidays. EBSP officials stated 
that some activities, such as testing, occur 
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to Which the Electronic Baggage- Screening- 
Program Schedule Meets Established Best 
Practices 



Best practice Explanation 



Criterion met 
(July 2010) 



Criterion met 
(October 2010) 



GAO analysis 



variances will affect 
subsequent work. 



continuously regardless of weekends or holidays and 
are, therefore, assigned to the 7-day workweek 
calendar. However, we found activities such as site 
design, equipment delivery, integration and 
installation, and documentation tasks scheduled to 
occur on weekends. 

Updated analysis: The updated schedule meets best 
practices for durations, with 94 percent of remaining 
activities being less than 44 days, and 99 percent less 
than 45 days. The schedule includes five calendars, 
four of which represent 8-hour, 5-day workweeks and 
account for holidays. However, the majority of 
remaining activities (61 percent) is based on a 7-day 
calendar that does not account for holidays. Activities 
based on a 7-day workweek with no holidays include 
tasks such as documentation, data analysis, training, 
and report deliveries. While it is possible that activities 
assigned to the 7-day calendar are related to activities 
that can run through holidays, some of the activities 
assigned to this calendar do not typically belong on a 
7-day calendar. 



5. Integrating 
schedule 
activities 
horizontally 
and vertically 



The schedule should 
be horizontally 
integrated, meaning 
that it should link 
products and 
outcomes associated 
with other sequenced 
activities. These links 
are commonly 
referred to as 
"handoffs" and serve 
to verify that activities 
are arranged in the 
right order to achieve 
aggregated products 
or outcomes. The 
schedule should also 
be vertically 
integrated, meaning 
that the dates for 
starting and 
completing activities 
in the integrated 
master schedule 
should be aligned 
with the dates for 
supporting tasks and 
subtasks. Such 
mapping or alignment 
among levels enables 
different groups to 



Partially met Partially met 



Initial Analysis: The schedule is vertically integrated, 
with low-level tasks and milestones being traceable to 
higher-level summary tasks. However, issues with 
sequencing logic prevent the schedule from being fully 
horizontal-integrated. Extending durations of some 
key activities had no effect on future milestone 
activities. For example, increasing the duration of the 
"Evaluate Pricing" activity associated with the first 
contract award from 40 days to 500 days has no effect 
on any of three contract award dates. Similarly, 
extending the duration of the first contracting activity, 
"Request decision to move forward with purchase" 
from 1 day to 90 days has no effect on any of the 
three contract award dates. 

Updated analysis: Issues remain with horizontal 
integration because extending the duration of some 
key activities to extremely long values does not affect 
future activities. For example, increasing the currently 
in-progress, non-critical task "[OT&E] Site Surveys" 
from 45 days to 500 days does not cause the task to 
become critical and has no effect on successor 
activities, including the first contract award date. 
Similarly, extending the durations of four activities 
related to Windows 1 and 2 Qualification Data 
Packages to 500 days each has no effect on future 
activities, including the first contract award. 
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to Which the Electronic Baggage- Screening- 
Program Schedule Meets Established Best 
Practices 



Best practice Explanation 



Criterion met 
(July 2010) 



Criterion met 
(October 2010) 



GAO analysis 



work to the same 
master schedule. 



Establishing 
the critical 
path for all 
activities 



Scheduling software 
should be used to 
identify the critical 
path, which 
represents the chain 
of dependent 
activities with the 
longest total duration. 
Establishing a 
project's critical path 
is necessary to 
examine the effects 
of any activity slipping 
along this path. 
Potential problems 
along or near the 
critical path should 
also be identified and 
reflected in 
scheduling the 
duration of high-risk 
activities. 



Minimally met Minimally met 



Initial analysis: Our analysis could not determine a valid 
critical path within the EBSP schedule due to missing 
logic links and incorrect statusing of activities. The 
EBSP schedule critical path begins via a data collection 
activity that is not logically linked to any predecessor 
activities, has a constrained start date, and is marked 
as 100 percent complete on June 18, 2010 — 2 months 
into the future according to the schedule status date of 
April 16, 2010. The critical path continues through two 
successor activities that are marked as having started 
several months into the future as well; then it continues 
through 24 successor activities until the third contract 
award milestone of December 12, 201 1 . However, 
according to the schedule, 11 of these critical activities 
are 334 days behind schedule; 3 activities are 341 days 
behind schedule, and 10 are 477 days behind 
schedule. In other words, these critical activities are 
shown as between 1.3 and 1 .8 working years behind 
schedule, assuming a 5-day workweek calendar. As 
there are 612 calendar days left until the project 
completion date of December 19, 201 1 (approximately 
2.3 working years), negative float fewer than -300 days 
does not appear to be a realistic representation of the 
true status of these activities. 

Schedules should include complete logic that 
addresses the relationships between predecessor and 
successor activities because any activity can become 
critical under some circumstances. Without clear insight 
into a critical path at the project level, management will 
not be able to monitor critical or near-critical detail 
activities that may have a detrimental impact on 
downstream activities if delayed. 

Updated analysis: Our analysis could not determine a 
valid critical path for the updated schedule. The critical 
path begins with an equipment delivery activity 
scheduled for November 14, 2010 — 7 months into the 
future relative to the status date, and one month into 
the actual future relative to the actual date of this 
analysis. A given activity cannot start in the future - that 
is, activities cannot actually start one month from now. 
Additionally, the status date is the time of the last 
update to the schedule, yet the critical path starts 7 
months into the future relative to this status date. 
Therefore, the critical path starts at the wrong relative 
time, and the wrong actual time. The critical path 
continues through 25 activities, which range in float 
values from -230 days to -336 days. In other words, 
according to the new revised plan, the 26 activities 
driving the revised date for the first contract award are 
10 to 15 months behind schedule, assuming a 5-day 
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to Which the Electronic Baggage- Screening- 
Program Schedule Meets Established Best 
Practices 



Best practice Explanation 



Criterion met 
(July 2010) 



Criterion met 
(October 2010) 



GAO analysis 



workweek calendar. 



7. Identifying The schedule should Minimally met 
reasonable identify the float — the 
float between amount of time by 
activities which a predecessor 

activity can slip 

before the delay 

affects successor 

activities — so that a 

schedule's flexibility 

can be determined. 

As a general rule, 

activities along the 

critical path have the 

least float. Total float 

is the total amount of 

time by which an 

activity can be 

delayed without 

delaying the project's 

completion, if 

everything else goes 

according to plan. 



Minimally met Initial analysis: Our analysis found that float 

calculations within the schedule are not reliable 
because of missing logic links and the high number of 
date constraints. If the schedule is missing 
dependencies or if activities are linked incorrectly, 
float estimates will be miscalculated because float is 
directly related to the logical sequencing of events. 

For example, 112 remaining activities (22 percent) have 
more than 300 days of total float meaning that, 
according to the schedule, these activities can slip 
more than one work-year and not affect the finish 
milestone. This includes 25 remaining activities (5 
percent of all remaining activities) with 500 or more 
days of float. Furthermore, there are 125 remaining 
activities (25 percent) with negative float. Negative float 
implies that there is not enough time scheduled for the 
task. Of the activities with negative float, 98 remaining 
activities (20 percent) appear as more than 50 days 
behind schedule, including 40 activities (8 percent) that 
are 300 or more days behind schedule. As there are 
612 calendar days left until the project completion date 
of December 1 9, 201 1 , float greater than 300 days or 
fewer than -300 days does not appear to be a realistic 
representation of the true status of these activities. 

Without reliable float estimates management may be 
unable to allocate resources from non-critical activities 
to activities that cannot slip without affecting the 
project finish date. 

Updated analysis: The schedule continues to reflect 
unreasonable amounts of negative float. Program 
officials stated that the negative float is a result of 
changes in the acquisition strategy resulting in a 
change to activity start dates. However, 98 percent of 
all remaining activities have negative float, ranging 
from -336 to -90 days. Only five remaining activities 
have positive float, which ranges from 10 to 486 days. 



8. Conducting a A schedule risk 
schedule risk analysis should be 
analysis performed using 

statistical techniques 
to predict the level of 
confidence in meeting 
a project's completion 
date. This analysis 
focuses not only on 
critical path activities, 
but also on activities 
near the critical path 
since they can affect 
the project's status. 



Not met 



Not met 



Initial analysis: The program office has not performed 
a schedule risk analysis (SRA). Officials stated that 
because DHS does not require an SRA, TSA has no 
contracts in place to support conducting the analysis 
and no knowledgeable support is available within TSA 
to perform an SRA. Program officials stated that in 
their opinion, an SRA appears to be too time- 
consuming and that they have no plans to seek 
support to perform the analysis. 

A comprehensive schedule risk analysis is an essential 
tool for decision makers. An SRA can be used to 
determine a level of confidence in meeting the 
completion date or whether proper reserves have been 
incorporated into the schedule. An SRA will calculate 
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to Which the Electronic Baggage- Screening- 
Program Schedule Meets Established Best 
Practices 



Best practice Explanation 



Criterion met 
(July 2010) 



Criterion met 
(October 2010) 



GAO analysis 



schedule reserve, which can be set aside for those 
activities identified as high-risk. Without this reserve, 
the program faces the risk of delays to the scheduled 
completion date if any delays were to occur on critical 
path activities. However, if the schedule risk analysis is 
to be credible, the program must have a quality 
schedule that reflects reliable logic and clearly identifies 
the critical path — conditions that the schedule does not 
meet. 

Updated analysis: No change to initial assessment. 



Updating the 
schedule 
using logic 
and durations 
to determine 
dates 



The schedule should 
be continuously 
updated using logic 
and durations to 
determine realistic 
start and completion 
dates for program 
activities. The 
schedule should be 
analyzed 
continuously for 
variances to 
determine when 
forecasted 
completion dates 
differ from planned 
dates. This analysis 
is especially 
important for those 
variations that affect 
activities identified as 
being on a project's 
critical path and can 
affect a scheduled 
completion date. 



Minimally met Minimally met 



Initial analysis: EBSP officials stated that EBSP 
conducts weekly meetings regarding the schedule and 
updates the status accordingly. From these weekly 
meetings, officials generate weekly reports that identify 
key areas of concern with regard to schedule shifts and 
potential impacts on milestones. However, our analysis 
of the schedule shows 30 activities (6 percent) that 
should have started and finished according to the 
schedule status date, but do not have actual start or 
finish dates. In addition, as previously mentioned, the 
critical path in part consists of activities that that are 
marked as having started several months into the 
future. A status date denotes the date of the latest 
update to the schedule and thus defines the point in 
time at which completed work and remaining work are 
calculated. The schedule also contains 43 activities (9 
percent) with actual start and finish dates in the future 
relative to the schedule status date. 

EBSP officials stated that relevant scheduling 
guidance, such as program or agency directives that 
govern the creation, maintenance, structure, and 
statusing of the schedule, does not exist. However, the 
schedule should be continually monitored to determine 
when forecasted completion dates differ from the 
planned dates, which can be used to determine 
whether schedule variances will affect downstream 
work. Maintaining the integrity of the schedule logic is 
not only necessary to reflect true status, but is also 
required before conducting a schedule risk analysis. 

Updated analysis: The updated schedule has the same 
April 16, 2010, status date as the original schedule, 
which EBSP officials stated is an error that will be 
corrected in future versions. Officials stated that while 
the status date was not correctly updated, updates to 
scheduled activities have been made to reflect actual 
dates on a weekly basis. However, the incorrect status 
creates the impression that 34 activities have occurred 
in the future. The updated schedule records actual start 
and actual finish dates for all completed activities but 
one. This activity should have finished on April 2, 2009, 
but has no actual finish date. 
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to Which the Electronic Baggage- Screening- 
Program Schedule Meets Established Best 
Practices 



Source: GAO analysis of TSA information. 

Notes: 

Fully met: The program provided complete evidence that satisfies all of the entire criteria for the 
identified best practice. 

Partially met: The program provided evidence that satisfies about half of the criteria for the identified 
best practice. 

Minimally met: The program provided evidence that satisfies less than half of the criteria for the 
identified best practice. 

Not met: The program did not provide evidence that satisfies any of the criteria for the identified best 
practice. 

a ln its initial schedule provided to GAO in July 2010, TSA established planned dates for the first three 
EDS contract awards. However, TSA's revised schedule provided to GAO in October 2010 plans for 
activities only through the first EDS contract award scheduled for July 201 1 . 



Page 53 



GAO-1 1-740 Aviation Security 



Appendix IV: Comments from the 
Department of Homeland Security 



I'.S. Department of Homefaud Security 
Washington. DC 20528 

$Sj& Homeland 
l|p' Security 

July 6, 2011 



Stephen M. Lord 

Director, Homeland Security and Justice 
U.S. Government Accountability Office 
441 G Street, NW 
Washington, DC 20548 

Re: Draft Report GAO- 1 1 -740, "AVIATION SECURITY; TSA Has Enhanced its 

Explosives Detection Requirements for Checked Baggage, but Additional Screening 
Actions Are Needed" 

Dear Mr. Lord: 

Thank you for the opportunity to review and comment this draft report. The U.S. Department of 
Homeland Security (DHS) appreciates the U.S. Government Accountability Office's (GAO) 
work in planning and conducting its review and issuing this report. 

Because of the urgent need of the program to meet ongoing requirements, the Transportation 
Security Administration (TSA) began actively resolving many of the issues identified in the audit 
in parallel to the audit being conducted. Contracts used by TSA to meet ongoing needs for 
Explosives Detection Systems (EDS) will expire in 201 1 and must be replaced to enable TSA to 
continue meeting baggage screening requirements. TSA also awarded 41 facility modifications 
to 36 airports under the American Recovery and Reinvestment Act and fiscal year (FY) 2009 
funding. As a result, a critical need exists to meet airport schedules for installation of EDS into 
baggage handling systems that are currently being built. To meet these demands for equipment, 
TSA began modifying its strategy for the EDS competitive procurement in 2010. This 
modification removed data collection from the EDS procurement critical path and will enable 
TSA to achieve higher levels of detection for EDS more quickly than would have been achieved 
through the original procurement strategy. 

Additionally, TSA must procure and install large quantities of EDS equipment to replace the 
aging fleet of EDS. TSA began developing a recapitalization plan for EDS in 2010 with the goal 
of prioritizing projects and leveling out the recapitalization of EDS equipment while continuing 
to meet emerging operational requirements of ongoing airport projects. As the recapitalization 
effort moves forward, TSA will replace this equipment with systems possessing higher levels of 
detection. 

TSA's Strategy for Increasing Detection Capability 

In December 2007. TSA decided to suspend the full implementation of the 2005 standard, 
including an operational impact evaluation, because of a computer modeling effort known as 
■"Project Newton," which had the potential to modify the 2005 detection standard. When Project 
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Newton did not yield timely results, TSA published the 20 1 detection standard that was a 
requirement for the EDS Competitive Procurement Process planned for contract awards in 201 1 . 
TSA is currently taking steps to collect the operational data necessary to support the upgrade of 
deployed equipment based on 20 1 detection standards. This data collection effort to support 
the upgrade is planned to be completed in 201 1 . In addition, vendor proposals for upgrading the 
existing fleet to the 20 1 detection standard have been requested and contract awards for 
upgrades are planned to be complete by the end of 201 1 . While achieving deployment of the 
2010 detection standard, TSA is also working with the DHS Science and Technology (S&T) 
Directorate toward establishing a laboratory by the summer of 201 1 to support further vendor 
data collection and independent test and evaluation. 

It is important to note that the detection requirements are dynamic in nature due to continuous 
intelligence driven threat assessments. As noted in the GAO report, TSA's current EDS 
competitive procurement strategy separates the procurement into multiple windows; only the 
first window's schedule has been outlined at this time. Each phase will require an incremental 
increase in detection capability, which reduces program risk, increases opportunities for vendors 
to qualify their technology, and enables deployment of the most current detection capability 
possible to the field. Achieving all detection levels will require collaborative research and 
development on the part of DHS S&T, TSA, and the vendor community. 

Additionally, TSA is developing a plan to address Explosives Trace Detection (ETD) units to 
resolve secondary alarms during the checked baggage screening process. Deployed technology 
meets current detection needs but TSA may need to pursue technology solutions to address the 
next level of detection within the necessary timeframe. The initial planning stages of this effort 
have already been started, and TSA is actively coordinating with DHS S&T to identify available 
technologies. 

Vendor Communication 

TSA significantly changed the business model for procurement of checked baggage screening 
equipment from what has historically been a sole source environment to a competition driven 
environment resulting in significant improvements in communication with industry. The 
changes TSA implemented drove business model changes on behalf of the equipment 
manufacturers that are now forced to compete for contract awards. Throughout the competitive 
procurement, TSA Electronic Baggage Screening Program (EBSP) leadership worked closely 
with the vendors and provided numerous opportunities for industry to offer feedback to the 
program. These opportunities included an initial kick-off with all vendors (2009); multiple 
industry days (2009 and 201 0); individual classified sessions to review detection requirements 
(20 1 0); and the publication of advance copies of the EDS specifications (2009) to offer vendors 
the opportunity to comment on planned detection and operational requirements prior to the time 
the competitive procurement process began and finalization of the requirements. In addition, 
TSA leadership visited each vendor site in 2009 to discuss the upcoming procurement and 
concerns of the vendors, and to communicate TSA's procurement plans. 
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Asa result of this extensive engagement and feedback from industry subject-matter experts on 
the capabilities of EDS technology today, TSA revised the detection and operational 
requirements, as well as the procurement strategy and schedule, to promote competition arid to 
drive enhanced EDS capabilities. This included revising the acquisition schedule to move the 
release of the Request for Proposal after the testing process had been significantly completed to 
promote continued open communication between TSA and the vendor community. TSA's 
ability to adhere to defined plans and schedules relies partially with industry's ability to meet 
their own commitments. As the acquisition process continues to advance, vendor performance 
has not always been consistent with the commitments that were made. 

TSA learned of industry concerns through these engagements and made the following 
adjustments to ensure these concerns were addressed: 

- TSA implemented bailment/Other Transaction Agreements with each vendor to pay for 
engineering support services throughout the testing and data collection processes. 

- The Operational Requirements Document was modified to reflect minimum and 
non-minimum requirements. 

- Additional time was added to the schedule to support Operational Test and Evaluation 
after completion of certification testing and integration testing. 

- Data was collected on each system submitted by each vendor for the procurement to 
support algorithm development. 

- Multiple windows of opportunity to compete in the procurement were created to allow 
industry the opportunity to go through the certification and testing process multiple times 
and to promote competition. 

In conclusion, TSA believes that its modified strategy for the EDS competitive procurement will 
increase detection capability in the most achievable, cost-effective approach possible. The 
Agency has taken a number of steps to address the issues discussed in this report and is actively 
developing plans to ensure that airports across the country are operating with the latest detection 
capability. Additionally, TSA has been responsive to industry, and extensive communication has 
taken place to promote competition; communicate TSA goals, objectives, and requirements; and 
to ensure success of both TSA and industry in the competitive procurement process. 

Recommendations 

The draft report contained six recommendations. As discussed below, DHS concurs with all six 
recommendations. Specifically, to help ensure that TSA takes a comprehensive and cost 
effective approach to the procurement and deployment of EDSs that meet the 2010 EDS 
requirements and any subsequent revisions, GAO recommended that the Assistant Secretary for 
TSA take the following actions: 

Recommendation I: Develop a plan to ensure that screening devices or protocols are in place 
to resolve EDS alarms if EDS are deployed that detect a broader set of explosives than existing 
ETD machines used to resolve EDS screening alarms. 
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Response: Concur. TSA recognizes that evolving detection standards for primary EDS 
screening may require an enhanced portfolio of technologies or procedures to support secondary 
threat resolution. TSA will be able to meet current detection needs but will pursue technology 
solutions to address the next level of detection requirements within the necessary timeframe. 
TSA convened a working group to assess capability gaps for secondary screening technology, 
evaluate current technology capabilities against the capabilities of future EDS, and prepare a plan 
to procure any additional technology required to ensure alarms can be resolved. This plan will 
be finalized by the fourth quarter of FY 2012. TSA is also collaborating with DBS S&T to drive 
research and development efforts toward the next level of trace detection technology. 

Recommendation 2: Develop a plan to ensure that TSA has the data needed for each of the 
planned phases of the 2010 EDS requirements before starting the procurement process for new 
EDS or upgrades included in each applicable phase. 

Response: Concur. TSA modified its strategy for the EDS competitive procurement in July 
2010 in response to challenges regarding safety issues and characteristics of the explosives used 
for data collection, and posted this change to the vendor community through Amendment 13 on 
September 3, 2010. The new baseline schedule outlines an incremental development process for 
detection that removes data collection from the acquisition process. Additionally, TSA is 
working with DHS S&T to establish a laboratory by summer 201 1 to support further vendor data 
collection and independent test and evaluation. 

Recommendation 3: Establish a process to communicate information to EDS vendors in a 
timely manner regarding TSA's EDS acquisition, including information such as changes to the 
schedule. 

Response: Concur. TSA has a process for communicating information to the vendor 
community and will continue to follow this process in adherence with guidelines outlined in the 
Federal Acquisition Regulation. The Agency has already made a number of efforts to improve 
the quality and frequency of communication with industry as detailed earlier. However, TSA 
recognizes the complexity associated with many of the acquisitions currently ongoing. As such, 
TSA acknowledges there are opportunities to continue to improve communications with the 
vendor community and looks forward to ensuring that vendors are provided with the most 
current information possible in an efficient manner. 

Recommendation 4: Develop and maintain an Integrated Master Schedule for the entire 
Electronic Baggage Screening Program in accordance with the nine best practices identified by 
GAO for preparing a schedule. 

Response; Concur. TSA has already begun working with key stakeholders to develop and 
define requirements for an Integrated Master Schedule (IMS) and ensure that our schedule is in 
alignment with the best practices outlined by GAO. This effort is expected to be completed by 
the second quarter of FY 2012. As the program matures and increases its focus on flexible and 
upgradable technology, an IMS will ensure close coordination between the program's 
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procurement, deployment, recapitalization, and upgrade capabilities. The EBSP IMS will be 
updated as a result of these efforts to be in accordance with the nine best practices. 

Recommendation 5: Ensure that key elements of the program's final cost estimate reflect 
critical issues, such as the potential cost impacts resulting from schedule slippage identified once 
an Integrated Master Schedule for the Electronic Baggage Screening Program has been 
developed in accordance with the nine best practices identified by GAO for preparing a schedule. 

Response: Concur. TSA agrees that the program's cost estimate should reflect potential cost 
increases and schedule risks associated with enhanced detection capabilities. TSA is working to 
update the EBSP life cycle cost estimate (LCCE) to incorporate cost estimates associated with 
enhanced detection, which should be complete in the fourth quarter of FY 201 1 . In addition, per 
recommendations of both GAO and DHS, TSA is also developing a master schedule to 
document timelines associated with various projects. Risks to the costs and schedules will be 
analyzed within the LCCE qualitatively in text and quantitatively via a scenario-based risk 
analysis approach recommended by DHS. The risk analysis will produce confidence intervals 
for the life cycle costs to the program. 

Recommendation 6: Develop a plan to deploy EDS that meets the most recent EDS explosives 
detection requirements and ensure that new machines, as well as machines deployed in airports, 
will be operated at the levels established in those requirements. This plan should include the 
estimated costs for new machines and upgrading deployed machines, and the timeframes for 
procuring and deploying new machines and upgrading deployed machines. 

Response: Concur. TSA has a plan in place to evaluate and implement the most recent certified 
algorithms on the existing fleet of deployed EDS, assuming the evaluation results has minimal to 
no operational impact. 

Again, thank you for the opportunity to review and comment on this draft report. Sensitivity and 
technical comments on the report have been provided under separate cover. We look forward to 
working with you on future Homeland Security engagements. 



Sincerely, 




^yn H. Crumpacker 
Director 

Departmental GAO/OIG Liaison Office 
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